-- Logs begin at Wed 2022-08-31 23:50:58 UTC, end at Thu 2022-09-01 00:51:43 UTC. -- Sep 01 00:15:49.030916 dv-cinder-820027-21-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 00:15:49.041021 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 00:15:49.042663 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 00:15:49 2022] *** Sep 01 00:15:49.042855 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 00:15:49.043030 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 00:15:49.043190 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: nodename: dv-cinder-820027-21-iscsi Sep 01 00:15:49.043344 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: machine: x86_64 Sep 01 00:15:49.043498 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: clock source: unix Sep 01 00:15:49.043646 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: pcre jit disabled Sep 01 00:15:49.043798 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: detected number of CPU cores: 4 Sep 01 00:15:49.043956 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: current working directory: / Sep 01 00:15:49.044108 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: detected binary path: /usr/bin/uwsgi-core Sep 01 00:15:49.049441 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: your processes number limit is 47680 Sep 01 00:15:49.049441 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: your memory page size is 4096 bytes Sep 01 00:15:49.049441 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: detected max file descriptor number: 2048 Sep 01 00:15:49.049441 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: lock engine: pthread robust mutexes Sep 01 00:15:49.049441 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: thunder lock: enabled Sep 01 00:15:49.049441 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 00:15:49.049441 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 00:15:49.084921 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: Python main interpreter initialized at 0x5563c1c289c0 Sep 01 00:15:49.084921 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: python threads support enabled Sep 01 00:15:49.084921 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: your server socket listen backlog is limited to 100 connections Sep 01 00:15:49.084921 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: your mercy for graceful operations on workers is 90 seconds Sep 01 00:15:49.085323 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 00:15:49.085403 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: *** Operational MODE: preforking *** Sep 01 00:15:49.085654 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: *** uWSGI is running in multiple interpreter mode *** Sep 01 00:15:49.085654 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: spawned uWSGI master process (pid: 93096) Sep 01 00:15:49.085871 dv-cinder-820027-21-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 00:15:49.086504 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: spawned uWSGI worker 1 (pid: 93105, cores: 1) Sep 01 00:15:49.087324 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: spawned uWSGI worker 2 (pid: 93106, cores: 1) Sep 01 00:15:49.087494 dv-cinder-820027-21-iscsi devstack@keystone.service[93096]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 00:15:53.446692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 00:15:53.446692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: from cryptography.utils import int_from_bytes Sep 01 00:15:53.447478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 00:15:53.447478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: from cryptography.utils import int_from_bytes Sep 01 00:15:53.447612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 00:15:53.447612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: from cryptography.utils import int_from_bytes Sep 01 00:15:53.447612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 00:15:53.447612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: from cryptography.utils import int_from_bytes Sep 01 00:15:53.497582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 00:15:53.497736 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 00:15:53.497821 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 00:15:53.497905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 00:15:53.497983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] command line args: [] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 00:15:53.498066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] command line args: [] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 00:15:53.498143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 00:15:53.498220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 00:15:53.498298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ================================================================================ {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 00:15:53.498376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ================================================================================ {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 00:15:53.498553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] admin_token = **** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.498645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] admin_token = **** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.498801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] config_dir = [] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.498906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] config_dir = [] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.498985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.499063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.499165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] config_source = [] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.499254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] config_source = [] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.499351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] debug = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.499431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] debug = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.499550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.499867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] list_limit = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.500958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] list_limit = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] log_config_append = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] log_config_append = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] log_dir = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] log_dir = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] log_file = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] log_file = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.501967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.502067 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.502176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.502272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.502374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.502476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.502587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.502699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.502803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.502913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.503902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.504002 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.504106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.504208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.504320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.504425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.504576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.504679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.504820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.504934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.505313 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.505425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.505532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] publish_errors = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.505634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] publish_errors = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.505742 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.505838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.505944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] standard_threads = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] standard_threads = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.506989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] use_journal = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] use_journal = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] use_json = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] use_json = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.507900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] use_stderr = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.508470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] use_stderr = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.508662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] use_syslog = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.508815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] use_syslog = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.508896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.509059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 00:15:53.509242 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.509327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.509491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.509571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.509755 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.509845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.509996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.510074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.510248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.510329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.510489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.510574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.510737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.510815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.510974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.511059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.511222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.511306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.511471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.511548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.511720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.511799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.511956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.512032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.512203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.512287 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.512439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.512554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.512718 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.512796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.512965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.513042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.513201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.513287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.513455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.513532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.513707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.513862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.513938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.514007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.514180 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.514259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.514431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.514516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.514692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.514777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.514943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.515026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.515193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.515279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.515423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.515500 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.515653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.515729 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.515882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.515968 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.516118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.516196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.516366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.516442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.516636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.516732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.516867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.516945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.517097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.517173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.517343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.517427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.517589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.517667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.517825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.517902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.518073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.518157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.518303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.518379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.518537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.518614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.518772 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.518860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519002 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.519997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.520164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.520282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.520383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.520477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.520644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.520750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.520878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.520995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.521103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.521183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.521339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.521421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.521589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.521678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.521823 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.521901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.522060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.522141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.522296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.522375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.522531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.522610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.522760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.522839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.522990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.523072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.523223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.523303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.523472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.523553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.524272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.524396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.524548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.524817 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.524893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.524975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.525047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.525165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.525253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.525406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.525487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.525648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.525730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.525891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.525982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.526135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.526218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.526408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.526503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.526652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.526746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.526892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.526980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.527134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.527217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.527371 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.527462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.527608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.527689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.527846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.527928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.528082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.528170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.528321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.528403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.528580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.528663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.528820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.528914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.529058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.529158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.529291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.529375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.529532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.529623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.529770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.529857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.530004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.530085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.530238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.530327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.530478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.530560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.530727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.530813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.530968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.531060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.531207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.531294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.531445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.531528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.531682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.531841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.531906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.531964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.532141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.532225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.532379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.532469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.532636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.532716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.532867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.532948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.533097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.533178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.533328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.533414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.533560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.533641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.533787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.533868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.534028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.534109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.534261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.534342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.534510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.534590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.534746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.534827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.535003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.535085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.535245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.535394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.535504 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] auth.external = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.535731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.535815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.535878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] auth.external = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.536033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.536121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.536252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.536385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.536471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] auth.password = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.536641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.536722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] auth.token = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.536858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] auth.password = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.536938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.537064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] auth.token = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.537166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.537301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.537381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.537542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.537626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.537793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.537884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.538022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.538102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.538272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.538355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.538508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] credential.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.538600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.538734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.538813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.538974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] credential.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.539057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.539249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.539346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.539507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.539755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.539894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.539973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.540132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.540229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.540385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.540468 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.540660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.540744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.540901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.540994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.541154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.541238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.541412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.541495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.541653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.541745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.541911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.541995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.542190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.542281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.542421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] federation.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.542510 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.542662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.542745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.542911 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.542994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.543161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.543245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.543394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.543477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] federation.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.543632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.543714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.543862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.543942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.544118 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.544200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.544351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.544807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.544893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.544957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545186 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.545867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.546006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.546070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.546245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.546329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.546492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.546583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.546724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.546803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.546956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.547041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.547192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.547274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.547443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.547527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.547674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.547759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.547906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.547990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.548132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.548213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.548425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.548533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.548677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.548759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.548903 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.548980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.549106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.549234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.549366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.549494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.549631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.549800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.549932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.550061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.550190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.550337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.550337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.550593 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.550722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.550851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.550978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.551106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.551243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.551372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.551501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.551629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.551757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.551884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.552030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.552160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.552288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.552416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.552590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.552734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.552865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.552993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.553120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.553248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.553376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.553513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.553643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.553771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.553908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.554043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.554172 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.554307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.554443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.554573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.554701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.554829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.554957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.555092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.555221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.555350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.555479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.555623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.555753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.555889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.556019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.556147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.556275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.556403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.556552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.556699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.556831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.556959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.557087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.557215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.557343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.557490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.557621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.557757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.557890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.558018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.558146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.558282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.558412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.558540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.558669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.558798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.558926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.559062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.559192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.559329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.559458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.559586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.559715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.559856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.559986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.560114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.560243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.560371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.560520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.560668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.560799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.560933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.561072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.561201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.561329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.561464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.561593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.561727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.561863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.561993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.562121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.562257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.562386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.562739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.562868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.562997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.563136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.563273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.563403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.563531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.563659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.563787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.563915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.564052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.564182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.564317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.564446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.564612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.564742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.564881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.565022 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.565152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.565281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.565409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.565537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.565678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.565808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.565936 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.566064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.566193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.566322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.566458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.566588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.566726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.566855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.566983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.567112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.567249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.567379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.567508 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.567637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.567855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.567990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.568128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.568259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.568394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.568543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] resource.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.568688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.568817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.568958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.569089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.569217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.569346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.569475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.569604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.569746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] resource.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.569877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.570006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.570135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.570263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.570401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.571109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.571248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.571380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.571509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.571638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.571844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] role.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.571977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.572107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] role.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.572236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.572365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.572589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.572753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.572887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.573016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.573157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] role.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.573286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.573415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] role.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.573553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.573683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.573812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.573941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.574069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.574197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.574334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.574464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.574595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.574723 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.574872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.575001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.575138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.575268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.575397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.575527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.575657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.575786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.575922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.576052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.576052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.576358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.576490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.576687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.576828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.576960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.577089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.577217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.577217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.577461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.577598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.577734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.577868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.577997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.578126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.578263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.578402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.578531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.578659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.578788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.578918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.579055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.579185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.579314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.579443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.579572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.579701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.579839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.579969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.580107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.580236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.580365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.580509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.580656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.580788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.580917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.581046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.581174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.581302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] token.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.581437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.581567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.581696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.581824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.581963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.582093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.582230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.582360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.582488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] token.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.582616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.582744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.582873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.583010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.583140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.583268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.583397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.583525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.583654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.583800 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.583931 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.584059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.584187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.584316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.584444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.584649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.584781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.584910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.585040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.585168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.585297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.585439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.585593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.585724 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.585853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.585981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.586841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.connection = **** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.587023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.587166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.587297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.587435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.587574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.587704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.587834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.587964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.588093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.connection = **** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.588242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.588374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.588533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.588670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.588800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.588930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.589071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.589203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.589333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.589462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.589590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.589720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.589857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.589987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.590125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.590256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.590385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.590514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.590650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.590780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.590909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.591039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.591168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.591298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.591436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.591566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.591695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.591824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.591962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93106) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 00:15:53.592092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.592092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.592092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.592092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 00:15:53.592092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93105) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 00:15:53.615161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.615392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.615618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.616033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.616267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.616539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.624062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.626262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.653188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.653414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.654732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.654965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.706337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.706823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.755359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.755539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.756263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.756517 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.910890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.911120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.911343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.911489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.911597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.911722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.911804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.911943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.912023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.912161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.912378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.912617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.935535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.935666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.935750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.935844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.935979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.936079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.936203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.936284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.936422 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.936521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.936663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.936744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.936880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.936974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.947951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.948318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.960702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.960893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.960997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.961654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.961785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.961866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.962052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.962133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.962273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.962353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.962487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93105) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.962567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.962701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.962902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93106) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 00:15:53.979614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.979760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.979844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 00:15:53.979972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 00:15:53.980152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.980314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.980396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 00:15:53.980531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 00:15:53.980808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.980888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.981069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.981150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.981326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.981406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.981581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.981660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.990980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.991169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.991250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.991449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.991530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.991723 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.991806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.992009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:53.999325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:53.999516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.002261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.002443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.002549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.002729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.002832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.003013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.003105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.003297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.003379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.003583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.003662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.003843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.004000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.004109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.004279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.004379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.004578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.004662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.004860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.004938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.005132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.005210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.028550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.028651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.029985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.030068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.030269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.030360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.034833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.034944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.035117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.035228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.035401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.035512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.042081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.042164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.042370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.042449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.046354 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.046490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.046647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.046782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.046936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.047070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.047221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.047357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.047498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.047636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.047913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.048186 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.048458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.049641 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.049735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.050018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.066192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.066288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.066471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.066551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.070343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.070532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.073591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.073890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.073971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.074198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.078253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.078428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.081428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.081704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.084555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.084842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.084925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.085140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.085221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.085424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.085514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.085705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.085825 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.085994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.086152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.086301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.102894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.102999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.103175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.103257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.103443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.103538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.103714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.103796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.110543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.110674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.112207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.112339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.113775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.113858 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.114052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.114131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.116727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.116870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.117012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.117155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.117306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.117716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.117800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.117874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.118039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.118121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.129401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.129492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.129687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.129767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.129970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.130067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.130253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.130335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.130535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.130616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.130815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.130897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.131099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.131178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.147268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.147375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.150635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.151320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.153368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.154275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.154929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.155726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.156400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.156707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.157001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.157245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.157529 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.157814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.163502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.164924 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.165698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.166405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.168647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.168942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.169228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.169334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.169508 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.169627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.169915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.170194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.177472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.177764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.178050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.178202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.178493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.178775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.185023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93105) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.185311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.185594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.185807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93106) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 00:15:54.185889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.186093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.186173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.186378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.186460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.186657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.186737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.186931 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.187020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.187208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.187297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.187493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.187580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.187773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.187853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.188056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.188136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.188335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.188418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.188629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.188712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93105) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.188893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.189161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.189441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93106) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 00:15:54.224056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5563c1c289c0 pid: 93105 (default app) Sep 01 00:15:54.225402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5563c1c289c0 pid: 93106 (default app) Sep 01 00:15:54.227937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef86bc-a568-4da0-8314-b21a13441e7f None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:15:54.228282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef86bc-a568-4da0-8314-b21a13441e7f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:15:54.228642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef86bc-a568-4da0-8314-b21a13441e7f None None] PATH_INFO: `/v3` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:15:54.233016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Thu Sep 1 00:15:54 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:00.821435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd0c85-3a97-4844-bd9c-268b6f3b6400 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:00.821977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd0c85-3a97-4844-bd9c-268b6f3b6400 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:00.821977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd0c85-3a97-4844-bd9c-268b6f3b6400 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:00.826292 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:16:00.831543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:00.832068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:00.832068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:00.903011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: INFO dbcounter [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] Registered counter for database keystone Sep 01 00:16:00.921796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG oslo_db.sqlalchemy.engines [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93105) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 00:16:00.927874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writer thread running {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 00:16:01.107400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG passlib.handlers.bcrypt [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93105) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 00:16:01.107709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG passlib.handlers.bcrypt [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93105) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 00:16:01.165677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:01.167275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:01.304201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 00:16:01.591392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd0c85-3a97-4844-bd9c-268b6f3b6400 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:01.665217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: INFO dbcounter [None req-d6bd0c85-3a97-4844-bd9c-268b6f3b6400 None None] Registered counter for database keystone Sep 01 00:16:01.686984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG oslo_db.sqlalchemy.engines [None req-d6bd0c85-3a97-4844-bd9c-268b6f3b6400 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93106) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 00:16:01.693074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writer thread running {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 00:16:01.813410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd0c85-3a97-4844-bd9c-268b6f3b6400 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:01.817644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17785d-3f7c-45c7-b5cd-20970afc5cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:01.818260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17785d-3f7c-45c7-b5cd-20970afc5cdd admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:01.818538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17785d-3f7c-45c7-b5cd-20970afc5cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:01.818811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17785d-3f7c-45c7-b5cd-20970afc5cdd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:01.827176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17785d-3f7c-45c7-b5cd-20970afc5cdd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:01.879320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5c17785d-3f7c-45c7-b5cd-20970afc5cdd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 00:16:01.879602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17785d-3f7c-45c7-b5cd-20970afc5cdd admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:01.889275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 1 00:16:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:16:01.895381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:01.909829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddad11-ae9c-409a-85ae-eb8761e0439d None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:01.914846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb917a83-60c0-417e-b985-043b4fcb441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:01.915936 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb917a83-60c0-417e-b985-043b4fcb441a admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:01.916219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb917a83-60c0-417e-b985-043b4fcb441a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:01.916508 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb917a83-60c0-417e-b985-043b4fcb441a admin admin] PATH_INFO: `/v3/projects/f6a8f68ec23146108279fe93631f855a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:01.918091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb917a83-60c0-417e-b985-043b4fcb441a admin admin] RBAC: Authorizing `identity:get_project(project_id=f6a8f68ec23146108279fe93631f855a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:01.975137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cb917a83-60c0-417e-b985-043b4fcb441a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 00:16:01.975490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb917a83-60c0-417e-b985-043b4fcb441a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:01.981197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Thu Sep 1 00:16:01 2022] GET /identity/v3/projects/f6a8f68ec23146108279fe93631f855a => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:03.857310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277668f-5acd-4934-8b09-26886f8dfc5b None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:03.857875 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277668f-5acd-4934-8b09-26886f8dfc5b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:03.857875 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277668f-5acd-4934-8b09-26886f8dfc5b None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:03.861977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:03.867800 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2e7bb-8151-4e14-8011-30b32e4f08f3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:03.868573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2e7bb-8151-4e14-8011-30b32e4f08f3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:03.868850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2e7bb-8151-4e14-8011-30b32e4f08f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:03.888313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ced2e7bb-8151-4e14-8011-30b32e4f08f3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:03.889857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ced2e7bb-8151-4e14-8011-30b32e4f08f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:03.897531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 00:16:04.159862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277668f-5acd-4934-8b09-26886f8dfc5b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:04.172282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277668f-5acd-4934-8b09-26886f8dfc5b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:04.176467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4777c7-bbef-41c8-81cd-ff7f3dfcc697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:04.177088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4777c7-bbef-41c8-81cd-ff7f3dfcc697 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:04.177369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4777c7-bbef-41c8-81cd-ff7f3dfcc697 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:04.177644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4777c7-bbef-41c8-81cd-ff7f3dfcc697 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:04.185842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4777c7-bbef-41c8-81cd-ff7f3dfcc697 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:04.187628 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4777c7-bbef-41c8-81cd-ff7f3dfcc697 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:04.196716 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 1 00:16:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:16:04.201861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2e7bb-8151-4e14-8011-30b32e4f08f3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:04.211437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2e7bb-8151-4e14-8011-30b32e4f08f3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:04.215956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35cd606-0a31-4547-9055-96043fdfdacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:04.216617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cd606-0a31-4547-9055-96043fdfdacb admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:04.216894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cd606-0a31-4547-9055-96043fdfdacb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:04.217161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cd606-0a31-4547-9055-96043fdfdacb admin admin] PATH_INFO: `/v3/users/6789433c4ccf40c89d963e435a40ba65` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:04.218458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35cd606-0a31-4547-9055-96043fdfdacb admin admin] RBAC: Authorizing `identity:get_user(user_id=6789433c4ccf40c89d963e435a40ba65)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:04.220282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35cd606-0a31-4547-9055-96043fdfdacb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:04.221843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Thu Sep 1 00:16:04 2022] GET /identity/v3/users/6789433c4ccf40c89d963e435a40ba65 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:06.179118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4bab9-240f-4fe6-92df-f02d8d85f183 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:06.179660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4bab9-240f-4fe6-92df-f02d8d85f183 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:06.179660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4bab9-240f-4fe6-92df-f02d8d85f183 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:06.181046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:06.186427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09a202-526f-4b63-b6d2-3776b0ef3bda None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:06.186845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09a202-526f-4b63-b6d2-3776b0ef3bda None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:06.186978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09a202-526f-4b63-b6d2-3776b0ef3bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:06.205309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-9b09a202-526f-4b63-b6d2-3776b0ef3bda None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:06.206496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9b09a202-526f-4b63-b6d2-3776b0ef3bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:06.226224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b87d7-b4b6-4f20-8a5a-eb0fca79e871 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:06.226544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b87d7-b4b6-4f20-8a5a-eb0fca79e871 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:06.226766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b87d7-b4b6-4f20-8a5a-eb0fca79e871 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:06.228620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:16:06.233893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4601-4731-4142-ac53-517416181d1a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:06.234174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4601-4731-4142-ac53-517416181d1a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:06.234446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d4601-4731-4142-ac53-517416181d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:06.276788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG passlib.handlers.bcrypt [None req-560d4601-4731-4142-ac53-517416181d1a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93106) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 00:16:06.277179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG passlib.handlers.bcrypt [None req-560d4601-4731-4142-ac53-517416181d1a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93106) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 00:16:06.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:06.326506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-560d4601-4731-4142-ac53-517416181d1a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:06.328118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-560d4601-4731-4142-ac53-517416181d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:06.427612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:06.555522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09a202-526f-4b63-b6d2-3776b0ef3bda None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:06.579859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:06.661290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09a202-526f-4b63-b6d2-3776b0ef3bda None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:06.693832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d4601-4731-4142-ac53-517416181d1a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:06.695008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8427d32-5b20-49d8-b7cb-23b94b01a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:06.695651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8427d32-5b20-49d8-b7cb-23b94b01a52c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:06.695964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8427d32-5b20-49d8-b7cb-23b94b01a52c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:06.696239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8427d32-5b20-49d8-b7cb-23b94b01a52c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:06.707462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8427d32-5b20-49d8-b7cb-23b94b01a52c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f8427d32-5b20-49d8-b7cb-23b94b01a52c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:06.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:06.717751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:06.718808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:16:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:06.724150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8427d32-5b20-49d8-b7cb-23b94b01a52c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:06.795261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d4601-4731-4142-ac53-517416181d1a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:06.797539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:06.806556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:06.825366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8427d32-5b20-49d8-b7cb-23b94b01a52c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:06.833780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d752355-c271-412a-86ca-9ab66063c1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:06.834639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d752355-c271-412a-86ca-9ab66063c1a0 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:06.834917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d752355-c271-412a-86ca-9ab66063c1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:06.835189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d752355-c271-412a-86ca-9ab66063c1a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:06.860645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb796b9-a4e9-4480-abbc-107b762c9a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:06.861254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb796b9-a4e9-4480-abbc-107b762c9a5a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:06.861531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb796b9-a4e9-4480-abbc-107b762c9a5a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:06.861802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb796b9-a4e9-4480-abbc-107b762c9a5a None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:06.862473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb796b9-a4e9-4480-abbc-107b762c9a5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:06.862773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb796b9-a4e9-4480-abbc-107b762c9a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:06.864365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb796b9-a4e9-4480-abbc-107b762c9a5a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:06.877067 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:06.882653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb796b9-a4e9-4480-abbc-107b762c9a5a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:06.903438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d752355-c271-412a-86ca-9ab66063c1a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:06.905089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d752355-c271-412a-86ca-9ab66063c1a0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:06.982271 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb796b9-a4e9-4480-abbc-107b762c9a5a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:07.001850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 00:16:07.007883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d752355-c271-412a-86ca-9ab66063c1a0 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:07.010955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:07.017533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf05011-e647-44a3-851a-b68a045e7b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:07.018162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf05011-e647-44a3-851a-b68a045e7b2e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:07.018435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf05011-e647-44a3-851a-b68a045e7b2e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:07.018701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf05011-e647-44a3-851a-b68a045e7b2e None admin] PATH_INFO: `/v3/users/6789433c4ccf40c89d963e435a40ba65` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:07.019819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf05011-e647-44a3-851a-b68a045e7b2e None admin] RBAC: Authorizing `identity:get_user(user_id=6789433c4ccf40c89d963e435a40ba65)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:07.021306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf05011-e647-44a3-851a-b68a045e7b2e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:07.022835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:06 2022] GET /identity/v3/users/6789433c4ccf40c89d963e435a40ba65 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:07.028882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf05011-e647-44a3-851a-b68a045e7b2e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:07.043603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:07.105861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d752355-c271-412a-86ca-9ab66063c1a0 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:07.130438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf05011-e647-44a3-851a-b68a045e7b2e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:07.139288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce494b-21a8-474c-bed9-83c5a1bfbf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:07.140141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce494b-21a8-474c-bed9-83c5a1bfbf73 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:07.140418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce494b-21a8-474c-bed9-83c5a1bfbf73 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:07.140716 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce494b-21a8-474c-bed9-83c5a1bfbf73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:07.158898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce494b-21a8-474c-bed9-83c5a1bfbf73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:07.160402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce494b-21a8-474c-bed9-83c5a1bfbf73 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:07.165368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ddcd7-ebcb-4678-ac69-21413b89ea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:07.165989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ddcd7-ebcb-4678-ac69-21413b89ea1f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:07.166268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ddcd7-ebcb-4678-ac69-21413b89ea1f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:07.166538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ddcd7-ebcb-4678-ac69-21413b89ea1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:07.167647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ddcd7-ebcb-4678-ac69-21413b89ea1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:07.169107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ddcd7-ebcb-4678-ac69-21413b89ea1f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:07.170647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:07.176322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ddcd7-ebcb-4678-ac69-21413b89ea1f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-07ce494b-21a8-474c-bed9-83c5a1bfbf73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:16:07.179165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:07.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:16:07.181950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:16:07.181950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:16:07.181950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:07.181950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:16:07.186611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce494b-21a8-474c-bed9-83c5a1bfbf73 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:07.245210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:07.261424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:07.281918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ddcd7-ebcb-4678-ac69-21413b89ea1f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:07.291907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce494b-21a8-474c-bed9-83c5a1bfbf73 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:07.315544 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d5eea-6660-4d48-83cf-1d480aea22fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:07.316152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d5eea-6660-4d48-83cf-1d480aea22fb None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:07.316431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d5eea-6660-4d48-83cf-1d480aea22fb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:07.316737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d5eea-6660-4d48-83cf-1d480aea22fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:07.317462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d5eea-6660-4d48-83cf-1d480aea22fb None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.domain.id=default, user.id=6789433c4ccf40c89d963e435a40ba65) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:07.317771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d5eea-6660-4d48-83cf-1d480aea22fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:07.319212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d5eea-6660-4d48-83cf-1d480aea22fb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:07.325632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526c6bd-5685-4408-9ab3-eb7b06fc324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:07.325781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 1 00:16:07 2022] GET /identity/v3/role_assignments?user.id=6789433c4ccf40c89d963e435a40ba65&scope.domain.id=default&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:07.326244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526c6bd-5685-4408-9ab3-eb7b06fc324b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:07.326521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526c6bd-5685-4408-9ab3-eb7b06fc324b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:07.326794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526c6bd-5685-4408-9ab3-eb7b06fc324b None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:07.327514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526c6bd-5685-4408-9ab3-eb7b06fc324b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:07.327822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526c6bd-5685-4408-9ab3-eb7b06fc324b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:07.329280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526c6bd-5685-4408-9ab3-eb7b06fc324b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:07.349528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:09.710030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c10294-b078-4932-ade7-9291125d061a None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:09.711199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c10294-b078-4932-ade7-9291125d061a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:09.711199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c10294-b078-4932-ade7-9291125d061a None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:09.712776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:09.718113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248623e-f932-467a-b846-d35d591435c8 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:09.718955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248623e-f932-467a-b846-d35d591435c8 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:09.719441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248623e-f932-467a-b846-d35d591435c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:09.731787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af493c8-4da7-46b1-884b-58346d70e1b1 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:09.732381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af493c8-4da7-46b1-884b-58346d70e1b1 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:09.732881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af493c8-4da7-46b1-884b-58346d70e1b1 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:09.734405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 13/23] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:16:09.739377 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dd6bc-5ba1-4259-9d02-4113ffcb01e9 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:09.739901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dd6bc-5ba1-4259-9d02-4113ffcb01e9 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:09.740402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980dd6bc-5ba1-4259-9d02-4113ffcb01e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:09.753679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-c248623e-f932-467a-b846-d35d591435c8 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:09.754890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-c248623e-f932-467a-b846-d35d591435c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:09.774840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-980dd6bc-5ba1-4259-9d02-4113ffcb01e9 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:09.776453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-980dd6bc-5ba1-4259-9d02-4113ffcb01e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:09.868538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 11/24] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:09.873056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e399b0-d759-4fc0-adfd-0283ea964c51 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:09.873356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e399b0-d759-4fc0-adfd-0283ea964c51 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:09.873639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e399b0-d759-4fc0-adfd-0283ea964c51 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:09.874930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 12/25] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:09.880486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d838a38-ff82-449f-9e6d-001ff87ba2db None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:09.880784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d838a38-ff82-449f-9e6d-001ff87ba2db None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:09.881054 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d838a38-ff82-449f-9e6d-001ff87ba2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:09.897749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 14/26] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:09.915961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0d838a38-ff82-449f-9e6d-001ff87ba2db None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:09.917173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0d838a38-ff82-449f-9e6d-001ff87ba2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:09.977854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:09.984963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b0d40-e9ea-4321-94d2-025cc4766832 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:09.985219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b0d40-e9ea-4321-94d2-025cc4766832 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:09.985446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b0d40-e9ea-4321-94d2-025cc4766832 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:09.986986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 15/27] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:09.997734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c2322-a3d3-4815-bbd4-4346a99669db None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:09.998014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c2322-a3d3-4815-bbd4-4346a99669db None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:09.998286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c2322-a3d3-4815-bbd4-4346a99669db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:10.014623 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 13/28] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:10.029345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-327c2322-a3d3-4815-bbd4-4346a99669db None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:10.030541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-327c2322-a3d3-4815-bbd4-4346a99669db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:10.119872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 16/29] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:10.271286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d838a38-ff82-449f-9e6d-001ff87ba2db None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:10.344656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c2322-a3d3-4815-bbd4-4346a99669db None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:10.349502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:10.406839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d838a38-ff82-449f-9e6d-001ff87ba2db None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:10.475491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779fc9d-55d8-41ab-874a-d5ed62265269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:10.476865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779fc9d-55d8-41ab-874a-d5ed62265269 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:10.477605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779fc9d-55d8-41ab-874a-d5ed62265269 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:10.478095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779fc9d-55d8-41ab-874a-d5ed62265269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:10.502024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779fc9d-55d8-41ab-874a-d5ed62265269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:10.503465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779fc9d-55d8-41ab-874a-d5ed62265269 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-c779fc9d-55d8-41ab-874a-d5ed62265269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:16:10.526236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:10.527144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:16:10.527946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:16:10.527946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:16:10.527946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:10.530176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 14/30] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:10.533327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779fc9d-55d8-41ab-874a-d5ed62265269 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:10.575932 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:10.577377 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c2322-a3d3-4815-bbd4-4346a99669db None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:10.617478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7f81f-7cca-4eb7-b6af-f24dced679cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:10.618462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7f81f-7cca-4eb7-b6af-f24dced679cf None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:10.618966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7f81f-7cca-4eb7-b6af-f24dced679cf None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:10.619446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7f81f-7cca-4eb7-b6af-f24dced679cf None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:10.620427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7f81f-7cca-4eb7-b6af-f24dced679cf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:10.622192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7f81f-7cca-4eb7-b6af-f24dced679cf None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:10.654105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779fc9d-55d8-41ab-874a-d5ed62265269 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:10.697909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83954151-8b2d-49b8-a0e5-dd4e5a300956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:10.699001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954151-8b2d-49b8-a0e5-dd4e5a300956 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:10.699535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954151-8b2d-49b8-a0e5-dd4e5a300956 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:10.700053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83954151-8b2d-49b8-a0e5-dd4e5a300956 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:10.714044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83954151-8b2d-49b8-a0e5-dd4e5a300956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-83954151-8b2d-49b8-a0e5-dd4e5a300956 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:10.721026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:10.722376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:10.723157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:10.725809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 15/31] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:10.727470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83954151-8b2d-49b8-a0e5-dd4e5a300956 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:10.762008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 17/32] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 00:16:10 2022] POST /identity/v3/roles => generated 219 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:16:10.763011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7f81f-7cca-4eb7-b6af-f24dced679cf None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:10.845964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:10.897757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7f81f-7cca-4eb7-b6af-f24dced679cf None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:10.922270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:10.938999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83954151-8b2d-49b8-a0e5-dd4e5a300956 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:10.942079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4800ef-6218-4e20-90ba-39d2d2ab0af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:10.942957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4800ef-6218-4e20-90ba-39d2d2ab0af0 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:10.943440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4800ef-6218-4e20-90ba-39d2d2ab0af0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:10.943912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4800ef-6218-4e20-90ba-39d2d2ab0af0 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:10.945441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4800ef-6218-4e20-90ba-39d2d2ab0af0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:10.947290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4800ef-6218-4e20-90ba-39d2d2ab0af0 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:10.971683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 18/33] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 00:16:10 2022] POST /identity/v3/roles => generated 225 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:10.973347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4800ef-6218-4e20-90ba-39d2d2ab0af0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:11.004000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c894f-1aad-4721-8795-90cd5abbd169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:11.005226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c894f-1aad-4721-8795-90cd5abbd169 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.005226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c894f-1aad-4721-8795-90cd5abbd169 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.005226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c894f-1aad-4721-8795-90cd5abbd169 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.005867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c894f-1aad-4721-8795-90cd5abbd169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:11.006172 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c894f-1aad-4721-8795-90cd5abbd169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:11.008288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c894f-1aad-4721-8795-90cd5abbd169 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:11.024577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 16/34] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:11.025818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40419c2-16d2-4a66-92cd-887c0c9c4a46 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.026104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40419c2-16d2-4a66-92cd-887c0c9c4a46 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.026382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40419c2-16d2-4a66-92cd-887c0c9c4a46 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.028026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 17/35] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:11.032781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40419c2-16d2-4a66-92cd-887c0c9c4a46 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:11.125786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4800ef-6218-4e20-90ba-39d2d2ab0af0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:11.164946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:11.168874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40419c2-16d2-4a66-92cd-887c0c9c4a46 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:11.177065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec4f80-824b-416a-b735-a142d29b606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:11.178104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec4f80-824b-416a-b735-a142d29b606d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.178605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec4f80-824b-416a-b735-a142d29b606d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.179111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec4f80-824b-416a-b735-a142d29b606d None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.180584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec4f80-824b-416a-b735-a142d29b606d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:11.181103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec4f80-824b-416a-b735-a142d29b606d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:11.183307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec4f80-824b-416a-b735-a142d29b606d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:11.196282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 19/36] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:11.198080 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18360c4-c562-455b-b2d5-ca6400348345 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.198572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18360c4-c562-455b-b2d5-ca6400348345 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.199131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18360c4-c562-455b-b2d5-ca6400348345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.218439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b6da8-8abf-48fa-9a87-519a377a4e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:11.219087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b6da8-8abf-48fa-9a87-519a377a4e82 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.219361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b6da8-8abf-48fa-9a87-519a377a4e82 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.219628 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b6da8-8abf-48fa-9a87-519a377a4e82 None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.220256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f18360c4-c562-455b-b2d5-ca6400348345 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:11.220602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b6da8-8abf-48fa-9a87-519a377a4e82 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:11.222458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f18360c4-c562-455b-b2d5-ca6400348345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:11.222991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b6da8-8abf-48fa-9a87-519a377a4e82 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:11.267923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:11.314142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 20/37] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:11.320746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18360c4-c562-455b-b2d5-ca6400348345 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:11.385971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 18/38] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:16:11 2022] POST /identity/v3/projects => generated 301 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:11.390411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e04ea-0504-48e2-a146-d7e46d2890b1 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.390706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e04ea-0504-48e2-a146-d7e46d2890b1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.390988 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e04ea-0504-48e2-a146-d7e46d2890b1 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.392471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 19/39] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:11.398163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be450ad9-6509-41fd-a325-62dcaf071961 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.398644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be450ad9-6509-41fd-a325-62dcaf071961 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.399053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be450ad9-6509-41fd-a325-62dcaf071961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.420326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-be450ad9-6509-41fd-a325-62dcaf071961 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:11.421995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-be450ad9-6509-41fd-a325-62dcaf071961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:11.438520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18360c4-c562-455b-b2d5-ca6400348345 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:11.459577 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:11.483739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dccf49-5805-49bf-bf91-3e3ed280d4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:11.484784 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dccf49-5805-49bf-bf91-3e3ed280d4c5 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.485278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dccf49-5805-49bf-bf91-3e3ed280d4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.485753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dccf49-5805-49bf-bf91-3e3ed280d4c5 None admin] PATH_INFO: `/v3/users/6789433c4ccf40c89d963e435a40ba65` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.487149 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dccf49-5805-49bf-bf91-3e3ed280d4c5 None admin] RBAC: Authorizing `identity:get_user(user_id=6789433c4ccf40c89d963e435a40ba65)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:11.488941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dccf49-5805-49bf-bf91-3e3ed280d4c5 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:11.490937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 21/40] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:11 2022] GET /identity/v3/users/6789433c4ccf40c89d963e435a40ba65 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:11.500330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dccf49-5805-49bf-bf91-3e3ed280d4c5 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:11.524082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 20/41] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:11.624115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be450ad9-6509-41fd-a325-62dcaf071961 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:11.725084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dccf49-5805-49bf-bf91-3e3ed280d4c5 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:11.739362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be450ad9-6509-41fd-a325-62dcaf071961 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:11.761359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa5ae7-5cd0-4df5-93f9-f528073a0b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:11.761996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa5ae7-5cd0-4df5-93f9-f528073a0b58 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.762396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa5ae7-5cd0-4df5-93f9-f528073a0b58 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.762923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa5ae7-5cd0-4df5-93f9-f528073a0b58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.763779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa5ae7-5cd0-4df5-93f9-f528073a0b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:11.765276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa5ae7-5cd0-4df5-93f9-f528073a0b58 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:11.766842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 22/42] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:11.772156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:11.772432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa5ae7-5cd0-4df5-93f9-f528073a0b58 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:11.779859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343c435-4bd8-4bca-8d62-bf909d9d7803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:11.780727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343c435-4bd8-4bca-8d62-bf909d9d7803 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.781183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343c435-4bd8-4bca-8d62-bf909d9d7803 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.781654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343c435-4bd8-4bca-8d62-bf909d9d7803 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.782550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343c435-4bd8-4bca-8d62-bf909d9d7803 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:11.784363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343c435-4bd8-4bca-8d62-bf909d9d7803 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:11.800895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 21/43] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 00:16:11 2022] POST /identity/v3/roles => generated 223 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:11.801707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343c435-4bd8-4bca-8d62-bf909d9d7803 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:11.832992 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:11.891627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa5ae7-5cd0-4df5-93f9-f528073a0b58 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:11.918573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343c435-4bd8-4bca-8d62-bf909d9d7803 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:11.929710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d41f94-d2dc-4873-ace6-f1ad966ddf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:11.930488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41f94-d2dc-4873-ace6-f1ad966ddf58 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.930774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41f94-d2dc-4873-ace6-f1ad966ddf58 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.931066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d41f94-d2dc-4873-ace6-f1ad966ddf58 None admin] PATH_INFO: `/v3/domains/default/users/6789433c4ccf40c89d963e435a40ba65/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.933134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d41f94-d2dc-4873-ace6-f1ad966ddf58 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6789433c4ccf40c89d963e435a40ba65, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:11.934623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d41f94-d2dc-4873-ace6-f1ad966ddf58 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:11.954352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 23/44] 10.222.0.25 () {58 vars in 1361 bytes} [Thu Sep 1 00:16:11 2022] PUT /identity/v3/domains/default/users/6789433c4ccf40c89d963e435a40ba65/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:16:11.956973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e19a9-e129-4a31-b4a7-426d069a3804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:11.957646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e19a9-e129-4a31-b4a7-426d069a3804 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:11.957918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e19a9-e129-4a31-b4a7-426d069a3804 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:11.958195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e19a9-e129-4a31-b4a7-426d069a3804 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:11.959392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e19a9-e129-4a31-b4a7-426d069a3804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:11.961217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e19a9-e129-4a31-b4a7-426d069a3804 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:11.962914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 22/45] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:11.969136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d41f94-d2dc-4873-ace6-f1ad966ddf58 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:12.080132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d41f94-d2dc-4873-ace6-f1ad966ddf58 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:12.094786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:12.118535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda724f-7a75-487c-bac1-0e5b25f05384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:12.119151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda724f-7a75-487c-bac1-0e5b25f05384 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:12.119433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda724f-7a75-487c-bac1-0e5b25f05384 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:12.119708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda724f-7a75-487c-bac1-0e5b25f05384 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:12.120359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda724f-7a75-487c-bac1-0e5b25f05384 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:12.121867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda724f-7a75-487c-bac1-0e5b25f05384 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:12.149852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:16:11 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:13.879409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0142c9f-2540-4a02-abed-7dcb86d274c1 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:13.879930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0142c9f-2540-4a02-abed-7dcb86d274c1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:13.880038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0142c9f-2540-4a02-abed-7dcb86d274c1 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:13.881490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 23/47] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:13.886744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c3b5c-0aa3-45be-b0d6-333884728da8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:13.887170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c3b5c-0aa3-45be-b0d6-333884728da8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:13.887464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c3b5c-0aa3-45be-b0d6-333884728da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:13.906426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-549c3b5c-0aa3-45be-b0d6-333884728da8 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:13.907627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-549c3b5c-0aa3-45be-b0d6-333884728da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:13.973687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 25/48] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:14.031588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26526df-c30a-4af4-80db-5f0cbbf1e7ee None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:14.031876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26526df-c30a-4af4-80db-5f0cbbf1e7ee None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:14.032159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26526df-c30a-4af4-80db-5f0cbbf1e7ee None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:14.033548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 24/49] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:14.039258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009962a-3dfe-48a0-817d-3b83e0029dad None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:14.039606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009962a-3dfe-48a0-817d-3b83e0029dad None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:14.039886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009962a-3dfe-48a0-817d-3b83e0029dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:14.057614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-4009962a-3dfe-48a0-817d-3b83e0029dad None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:14.058787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4009962a-3dfe-48a0-817d-3b83e0029dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:14.122240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 26/50] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:14.238155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26526df-c30a-4af4-80db-5f0cbbf1e7ee None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:14.298157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:14.343034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26526df-c30a-4af4-80db-5f0cbbf1e7ee None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:14.377081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84e5b-b8d6-4825-a88f-22a6ecfa7100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:14.377712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84e5b-b8d6-4825-a88f-22a6ecfa7100 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:14.377993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84e5b-b8d6-4825-a88f-22a6ecfa7100 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:14.378267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84e5b-b8d6-4825-a88f-22a6ecfa7100 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:14.387707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009962a-3dfe-48a0-817d-3b83e0029dad None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:14.389514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b84e5b-b8d6-4825-a88f-22a6ecfa7100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-d1b84e5b-b8d6-4825-a88f-22a6ecfa7100 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:14.394602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:14.395473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:14.396224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 25/51] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:16:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:14.403617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84e5b-b8d6-4825-a88f-22a6ecfa7100 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:14.413771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:14.494380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009962a-3dfe-48a0-817d-3b83e0029dad None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:14.531000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9760cf-8e23-44a6-8853-481cb458cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:14.531659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9760cf-8e23-44a6-8853-481cb458cc64 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:14.531947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9760cf-8e23-44a6-8853-481cb458cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:14.532217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9760cf-8e23-44a6-8853-481cb458cc64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:14.533389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9760cf-8e23-44a6-8853-481cb458cc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:14.534863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9760cf-8e23-44a6-8853-481cb458cc64 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:14.536455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 27/52] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:14.542048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9760cf-8e23-44a6-8853-481cb458cc64 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:14.610046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84e5b-b8d6-4825-a88f-22a6ecfa7100 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:14.626787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:14.633782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:14.642906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9760cf-8e23-44a6-8853-481cb458cc64 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:14.644642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f361303-019c-4344-86df-285c0ae6c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:14.645260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f361303-019c-4344-86df-285c0ae6c174 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:14.645538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f361303-019c-4344-86df-285c0ae6c174 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:14.645817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f361303-019c-4344-86df-285c0ae6c174 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:14.646487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f361303-019c-4344-86df-285c0ae6c174 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:14.646792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f361303-019c-4344-86df-285c0ae6c174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:14.648452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f361303-019c-4344-86df-285c0ae6c174 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:14.660308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 26/53] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:14.666625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f361303-019c-4344-86df-285c0ae6c174 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:14.675825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b318e-a210-483c-a4c1-6b2a6a2a6946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:14.676427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b318e-a210-483c-a4c1-6b2a6a2a6946 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:14.676736 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b318e-a210-483c-a4c1-6b2a6a2a6946 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:14.677008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b318e-a210-483c-a4c1-6b2a6a2a6946 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:14.677648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b318e-a210-483c-a4c1-6b2a6a2a6946 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:14.679194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b318e-a210-483c-a4c1-6b2a6a2a6946 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:14.705194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 28/54] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 1 00:16:14 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:16:14.773116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f361303-019c-4344-86df-285c0ae6c174 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:14.808233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b73df-5eb9-4544-8b77-1b33e7b32209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:14.808927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b73df-5eb9-4544-8b77-1b33e7b32209 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:14.809206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b73df-5eb9-4544-8b77-1b33e7b32209 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:14.809480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b73df-5eb9-4544-8b77-1b33e7b32209 None admin] PATH_INFO: `/v3/users/6789433c4ccf40c89d963e435a40ba65` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:14.810905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b73df-5eb9-4544-8b77-1b33e7b32209 None admin] RBAC: Authorizing `identity:get_user(user_id=6789433c4ccf40c89d963e435a40ba65)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:14.812394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b73df-5eb9-4544-8b77-1b33e7b32209 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:14.814046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 27/55] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:14 2022] GET /identity/v3/users/6789433c4ccf40c89d963e435a40ba65 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:14.819946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b318e-a210-483c-a4c1-6b2a6a2a6946 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:14.921189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b318e-a210-483c-a4c1-6b2a6a2a6946 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:14.955074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbc680-9748-4610-bdde-4dddc5a7ae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:14.955707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbc680-9748-4610-bdde-4dddc5a7ae6b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:14.955989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbc680-9748-4610-bdde-4dddc5a7ae6b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:14.956263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbc680-9748-4610-bdde-4dddc5a7ae6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:14.957387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbc680-9748-4610-bdde-4dddc5a7ae6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:14.958862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbc680-9748-4610-bdde-4dddc5a7ae6b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:14.960680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 29/56] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:14.967159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b73df-5eb9-4544-8b77-1b33e7b32209 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:15.020886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:15.072239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b73df-5eb9-4544-8b77-1b33e7b32209 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:15.107904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6931221-fcd1-4a9a-b59e-e449b6baf31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:15.108528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6931221-fcd1-4a9a-b59e-e449b6baf31d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:15.108818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6931221-fcd1-4a9a-b59e-e449b6baf31d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:15.109095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6931221-fcd1-4a9a-b59e-e449b6baf31d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:15.109820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6931221-fcd1-4a9a-b59e-e449b6baf31d None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.domain.id=default, user.id=6789433c4ccf40c89d963e435a40ba65) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:15.110131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6931221-fcd1-4a9a-b59e-e449b6baf31d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:15.111597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6931221-fcd1-4a9a-b59e-e449b6baf31d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:15.120872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 28/57] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 1 00:16:14 2022] GET /identity/v3/role_assignments?user.id=6789433c4ccf40c89d963e435a40ba65&scope.domain.id=default&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:16.616981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ba78a-21fa-4aa4-959b-f20fe1b8c64f None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:16.617608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ba78a-21fa-4aa4-959b-f20fe1b8c64f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:16.617771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ba78a-21fa-4aa4-959b-f20fe1b8c64f None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:16.619125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 30/58] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:16.624391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f58c63-c9a8-4d2b-8d7a-8618e16f6fc9 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:16.624828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f58c63-c9a8-4d2b-8d7a-8618e16f6fc9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:16.624979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f58c63-c9a8-4d2b-8d7a-8618e16f6fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:16.643910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-93f58c63-c9a8-4d2b-8d7a-8618e16f6fc9 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:16.645129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-93f58c63-c9a8-4d2b-8d7a-8618e16f6fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:16.715947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 29/59] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:16.982289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ba78a-21fa-4aa4-959b-f20fe1b8c64f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:16.997738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:17.088361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ba78a-21fa-4aa4-959b-f20fe1b8c64f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:17.122195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7e2ea-ed37-4ce2-91f9-2aaf9a18614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:17.122822 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7e2ea-ed37-4ce2-91f9-2aaf9a18614d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:17.123101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7e2ea-ed37-4ce2-91f9-2aaf9a18614d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:17.123375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7e2ea-ed37-4ce2-91f9-2aaf9a18614d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:17.124478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7e2ea-ed37-4ce2-91f9-2aaf9a18614d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:17.125956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7e2ea-ed37-4ce2-91f9-2aaf9a18614d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:17.127508 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 31/60] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:17.133665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f58c63-c9a8-4d2b-8d7a-8618e16f6fc9 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:17.174039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:17.228557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f58c63-c9a8-4d2b-8d7a-8618e16f6fc9 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:17.262745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4860db-6d2d-43ed-9a22-4a9ad5e4d002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:17.263290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4860db-6d2d-43ed-9a22-4a9ad5e4d002 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:17.263568 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4860db-6d2d-43ed-9a22-4a9ad5e4d002 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:17.263840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4860db-6d2d-43ed-9a22-4a9ad5e4d002 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:17.264515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4860db-6d2d-43ed-9a22-4a9ad5e4d002 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:17.265980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4860db-6d2d-43ed-9a22-4a9ad5e4d002 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:17.292324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 30/61] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:16:17 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:20.349785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e50042-8e82-4268-9f4b-07f802b8dc82 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:20.351277 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e50042-8e82-4268-9f4b-07f802b8dc82 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:20.351751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e50042-8e82-4268-9f4b-07f802b8dc82 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:20.353875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 32/62] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:20.359150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244c3c7-544a-462a-a2de-10b504ef8993 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:20.360001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244c3c7-544a-462a-a2de-10b504ef8993 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:20.360473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244c3c7-544a-462a-a2de-10b504ef8993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:20.384986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3244c3c7-544a-462a-a2de-10b504ef8993 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:20.386573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3244c3c7-544a-462a-a2de-10b504ef8993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:20.399135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fff70-fcb1-46fb-af86-e59b2b9e7f85 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:20.399714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fff70-fcb1-46fb-af86-e59b2b9e7f85 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:20.400207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fff70-fcb1-46fb-af86-e59b2b9e7f85 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:20.402026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 33/63] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:20.413352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aa8c7-bf62-4805-a637-8001741be044 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:20.413963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aa8c7-bf62-4805-a637-8001741be044 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:20.414514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aa8c7-bf62-4805-a637-8001741be044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:20.439109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ac3aa8c7-bf62-4805-a637-8001741be044 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:20.441426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ac3aa8c7-bf62-4805-a637-8001741be044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:20.507512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 31/64] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:20.559801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 34/65] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:20.594312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d966f-2839-46af-b6a0-8e045e6dfcc8 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:20.594939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d966f-2839-46af-b6a0-8e045e6dfcc8 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:20.595425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d966f-2839-46af-b6a0-8e045e6dfcc8 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:20.597021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 32/66] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:20.603271 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd5e7e-856d-4bc9-b612-9e7d2be23baf None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:20.603562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd5e7e-856d-4bc9-b612-9e7d2be23baf None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:20.603844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd5e7e-856d-4bc9-b612-9e7d2be23baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:20.633792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5edd5e7e-856d-4bc9-b612-9e7d2be23baf None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:20.634995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5edd5e7e-856d-4bc9-b612-9e7d2be23baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:20.641492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:20.733194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 35/67] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:20.858071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d966f-2839-46af-b6a0-8e045e6dfcc8 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:20.895538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:20.952919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd5e7e-856d-4bc9-b612-9e7d2be23baf None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.003906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d966f-2839-46af-b6a0-8e045e6dfcc8 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.041795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3f109-5edd-4579-8f8c-1307911bff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.042430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3f109-5edd-4579-8f8c-1307911bff0e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.042715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3f109-5edd-4579-8f8c-1307911bff0e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.042996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3f109-5edd-4579-8f8c-1307911bff0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.054503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3f109-5edd-4579-8f8c-1307911bff0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-cfa3f109-5edd-4579-8f8c-1307911bff0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:21.060363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:21.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:21.062490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 33/68] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:16:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:21.063833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247c648-65fa-47aa-ac44-62bd96b4ddd4 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.064128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247c648-65fa-47aa-ac44-62bd96b4ddd4 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.064408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247c648-65fa-47aa-ac44-62bd96b4ddd4 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.065867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 34/69] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:21.066756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1fca1-23c0-4c99-bb96-b94c8bac9c00 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.067040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1fca1-23c0-4c99-bb96-b94c8bac9c00 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.067327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1fca1-23c0-4c99-bb96-b94c8bac9c00 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.068635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 35/70] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:21.070473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1fca1-23c0-4c99-bb96-b94c8bac9c00 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.080790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd5e7e-856d-4bc9-b612-9e7d2be23baf None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.115172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38c18e-7317-431b-aa51-f39e6c264760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.115731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38c18e-7317-431b-aa51-f39e6c264760 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.116061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38c18e-7317-431b-aa51-f39e6c264760 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.116376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38c18e-7317-431b-aa51-f39e6c264760 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.123664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:21.126954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38c18e-7317-431b-aa51-f39e6c264760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-ea38c18e-7317-431b-aa51-f39e6c264760 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:21.132612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:21.133658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:21.134419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 36/71] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:16:21.135024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38c18e-7317-431b-aa51-f39e6c264760 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.168290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:21.181732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1fca1-23c0-4c99-bb96-b94c8bac9c00 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.216625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a15097-9b41-4cf3-bbbf-4650d0d9f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.217256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a15097-9b41-4cf3-bbbf-4650d0d9f3d4 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.217540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a15097-9b41-4cf3-bbbf-4650d0d9f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.217817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a15097-9b41-4cf3-bbbf-4650d0d9f3d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.229364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a15097-9b41-4cf3-bbbf-4650d0d9f3d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:21.233705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38c18e-7317-431b-aa51-f39e6c264760 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-d6a15097-9b41-4cf3-bbbf-4650d0d9f3d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:21.234647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:21.235490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:21.236239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 36/72] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:21.237386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d34c-abcc-4bf4-a9fc-911a383f3fe1 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.237386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d34c-abcc-4bf4-a9fc-911a383f3fe1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.237562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d34c-abcc-4bf4-a9fc-911a383f3fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.255867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-9ff8d34c-abcc-4bf4-a9fc-911a383f3fe1 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:21.257743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9ff8d34c-abcc-4bf4-a9fc-911a383f3fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:21.271954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a14b12-6870-4ff0-a804-9e472035a0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.273243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a14b12-6870-4ff0-a804-9e472035a0f3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.273398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a14b12-6870-4ff0-a804-9e472035a0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.273671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a14b12-6870-4ff0-a804-9e472035a0f3 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.274327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a14b12-6870-4ff0-a804-9e472035a0f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:21.274624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a14b12-6870-4ff0-a804-9e472035a0f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:21.276016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a14b12-6870-4ff0-a804-9e472035a0f3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:21.284176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 37/73] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:21.285387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b98db-cf9a-4e6a-98f8-51922fed1d00 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.285675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b98db-cf9a-4e6a-98f8-51922fed1d00 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.285950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b98db-cf9a-4e6a-98f8-51922fed1d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.304951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-276b98db-cf9a-4e6a-98f8-51922fed1d00 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:21.306164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-276b98db-cf9a-4e6a-98f8-51922fed1d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:21.324799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 37/74] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:21.325638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d34c-abcc-4bf4-a9fc-911a383f3fe1 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.375835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:21.387955 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 38/75] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:21.388812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b98db-cf9a-4e6a-98f8-51922fed1d00 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.418736 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:21.451492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d34c-abcc-4bf4-a9fc-911a383f3fe1 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.488044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864add04-ad61-42ce-8d21-f4a639146813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.488785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864add04-ad61-42ce-8d21-f4a639146813 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.489029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864add04-ad61-42ce-8d21-f4a639146813 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.489301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864add04-ad61-42ce-8d21-f4a639146813 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.489965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864add04-ad61-42ce-8d21-f4a639146813 None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:21.490264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864add04-ad61-42ce-8d21-f4a639146813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:21.491684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864add04-ad61-42ce-8d21-f4a639146813 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:21.504287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 38/76] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:21.504287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864add04-ad61-42ce-8d21-f4a639146813 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.517540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b98db-cf9a-4e6a-98f8-51922fed1d00 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.557470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e72fa-1e12-4ac2-84c4-21ffe27870d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.558110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e72fa-1e12-4ac2-84c4-21ffe27870d0 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.558384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e72fa-1e12-4ac2-84c4-21ffe27870d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.558679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e72fa-1e12-4ac2-84c4-21ffe27870d0 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.559313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e72fa-1e12-4ac2-84c4-21ffe27870d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:21.559609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e72fa-1e12-4ac2-84c4-21ffe27870d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:21.561093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e72fa-1e12-4ac2-84c4-21ffe27870d0 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:21.569863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 39/77] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:21.575098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e72fa-1e12-4ac2-84c4-21ffe27870d0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.625707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864add04-ad61-42ce-8d21-f4a639146813 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.638489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:21.664323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f3d47-600c-4a56-a665-744a98a1352e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.665021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f3d47-600c-4a56-a665-744a98a1352e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.665304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f3d47-600c-4a56-a665-744a98a1352e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.665576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f3d47-600c-4a56-a665-744a98a1352e None admin] PATH_INFO: `/v3/users/6789433c4ccf40c89d963e435a40ba65` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.667925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f3d47-600c-4a56-a665-744a98a1352e None admin] RBAC: Authorizing `identity:get_user(user_id=6789433c4ccf40c89d963e435a40ba65)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:21.669407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f3d47-600c-4a56-a665-744a98a1352e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:21.670990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 39/78] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/users/6789433c4ccf40c89d963e435a40ba65 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:21.671754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f3d47-600c-4a56-a665-744a98a1352e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.682970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:21.693579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e72fa-1e12-4ac2-84c4-21ffe27870d0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.728061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbeff6-3153-452c-a88e-7d5f1d6199f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.728721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbeff6-3153-452c-a88e-7d5f1d6199f8 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.729005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbeff6-3153-452c-a88e-7d5f1d6199f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.729280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbeff6-3153-452c-a88e-7d5f1d6199f8 None admin] PATH_INFO: `/v3/users/1a50181ffd4545db88d27e0331cd3f67` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.743235 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbeff6-3153-452c-a88e-7d5f1d6199f8 None admin] RBAC: Authorizing `identity:get_user(user_id=1a50181ffd4545db88d27e0331cd3f67)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:21.744740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbeff6-3153-452c-a88e-7d5f1d6199f8 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:21.746349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 40/79] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/users/1a50181ffd4545db88d27e0331cd3f67 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:21.751186 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbeff6-3153-452c-a88e-7d5f1d6199f8 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.773628 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f3d47-600c-4a56-a665-744a98a1352e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.806678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773f812-d367-4295-aff0-e35167ca0eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.807305 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773f812-d367-4295-aff0-e35167ca0eeb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.807590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773f812-d367-4295-aff0-e35167ca0eeb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.807864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773f812-d367-4295-aff0-e35167ca0eeb None admin] PATH_INFO: `/v3/users/1a50181ffd4545db88d27e0331cd3f67` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.809039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773f812-d367-4295-aff0-e35167ca0eeb None admin] RBAC: Authorizing `identity:get_user(user_id=1a50181ffd4545db88d27e0331cd3f67)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:21.810513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773f812-d367-4295-aff0-e35167ca0eeb None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:21.812099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 40/80] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/users/1a50181ffd4545db88d27e0331cd3f67 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:21.812896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773f812-d367-4295-aff0-e35167ca0eeb None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.855530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbeff6-3153-452c-a88e-7d5f1d6199f8 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.868846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:21.891619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdd136-4263-4b04-9ebc-081c8c241b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.892409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdd136-4263-4b04-9ebc-081c8c241b80 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.892602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:21.892907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdd136-4263-4b04-9ebc-081c8c241b80 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.893239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdd136-4263-4b04-9ebc-081c8c241b80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.894697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdd136-4263-4b04-9ebc-081c8c241b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:21.896352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cdd136-4263-4b04-9ebc-081c8c241b80 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:21.899019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 41/81] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:21.899795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdd136-4263-4b04-9ebc-081c8c241b80 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:21.916275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773f812-d367-4295-aff0-e35167ca0eeb None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:21.949842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c0cb2-99f3-4bfc-82f3-3fd40da85f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:21.950475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c0cb2-99f3-4bfc-82f3-3fd40da85f86 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:21.950755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c0cb2-99f3-4bfc-82f3-3fd40da85f86 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:21.951028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c0cb2-99f3-4bfc-82f3-3fd40da85f86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:21.961712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c0cb2-99f3-4bfc-82f3-3fd40da85f86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-a06c0cb2-99f3-4bfc-82f3-3fd40da85f86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:21.966909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:21.967922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 41/82] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:21.968848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c0cb2-99f3-4bfc-82f3-3fd40da85f86 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:22.002951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdd136-4263-4b04-9ebc-081c8c241b80 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:22.035673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac529692-d36c-4a04-808e-19fa75f12a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:22.036255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac529692-d36c-4a04-808e-19fa75f12a97 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:22.036568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac529692-d36c-4a04-808e-19fa75f12a97 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:22.036844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac529692-d36c-4a04-808e-19fa75f12a97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:22.067575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c0cb2-99f3-4bfc-82f3-3fd40da85f86 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:22.084337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:22.100043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be80446-df42-4ddf-9073-0a6be9b0b5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:22.101202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be80446-df42-4ddf-9073-0a6be9b0b5f9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:22.101477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be80446-df42-4ddf-9073-0a6be9b0b5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:22.101740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be80446-df42-4ddf-9073-0a6be9b0b5f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:22.104413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac529692-d36c-4a04-808e-19fa75f12a97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:22.105327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:22.105938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac529692-d36c-4a04-808e-19fa75f12a97 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:22.168648 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be80446-df42-4ddf-9073-0a6be9b0b5f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:22.170042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be80446-df42-4ddf-9073-0a6be9b0b5f9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:22.204991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 42/83] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:22.205838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac529692-d36c-4a04-808e-19fa75f12a97 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:22.270571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 42/84] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:22.275958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be80446-df42-4ddf-9073-0a6be9b0b5f9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:22.292960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:22.306531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:22.313319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac529692-d36c-4a04-808e-19fa75f12a97 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:22.346871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1ba87-5bdc-4bfe-be03-c7e398610026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:22.347478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1ba87-5bdc-4bfe-be03-c7e398610026 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:22.347759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1ba87-5bdc-4bfe-be03-c7e398610026 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:22.348031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1ba87-5bdc-4bfe-be03-c7e398610026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:22.373256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be80446-df42-4ddf-9073-0a6be9b0b5f9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:22.407823 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd252876-0617-4e41-be69-93c4b202208a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:22.408432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd252876-0617-4e41-be69-93c4b202208a None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:22.408738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd252876-0617-4e41-be69-93c4b202208a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:22.409014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd252876-0617-4e41-be69-93c4b202208a None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:22.409656 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd252876-0617-4e41-be69-93c4b202208a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:22.411115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd252876-0617-4e41-be69-93c4b202208a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:22.415365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1ba87-5bdc-4bfe-be03-c7e398610026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:22.416866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1ba87-5bdc-4bfe-be03-c7e398610026 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:22.431950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 43/85] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:16:22 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:22.432735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd252876-0617-4e41-be69-93c4b202208a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:22.518561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:22.537929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:22.543359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 43/86] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:22.548696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1ba87-5bdc-4bfe-be03-c7e398610026 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:22.550005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd252876-0617-4e41-be69-93c4b202208a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:22.591757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894ac37-81b7-4f1e-8c10-f7a218a540db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:22.592381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ac37-81b7-4f1e-8c10-f7a218a540db None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:22.593115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ac37-81b7-4f1e-8c10-f7a218a540db None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:22.594073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ac37-81b7-4f1e-8c10-f7a218a540db None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:22.594073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894ac37-81b7-4f1e-8c10-f7a218a540db None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:22.595125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894ac37-81b7-4f1e-8c10-f7a218a540db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:22.596230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894ac37-81b7-4f1e-8c10-f7a218a540db None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:22.606032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 44/87] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:22.607092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894ac37-81b7-4f1e-8c10-f7a218a540db None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:22.674931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1ba87-5bdc-4bfe-be03-c7e398610026 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:22.726145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4124c1-d856-43c1-a30a-3e51147a806f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:22.727109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4124c1-d856-43c1-a30a-3e51147a806f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:22.727397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4124c1-d856-43c1-a30a-3e51147a806f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:22.727682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4124c1-d856-43c1-a30a-3e51147a806f None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:22.728930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4124c1-d856-43c1-a30a-3e51147a806f None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:22.730399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4124c1-d856-43c1-a30a-3e51147a806f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:22.732210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 44/88] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:22 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:22.732940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4124c1-d856-43c1-a30a-3e51147a806f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:22.740635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894ac37-81b7-4f1e-8c10-f7a218a540db None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:22.786554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5350df-c3e5-4e3e-944d-e3b55b8e226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:22.787198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5350df-c3e5-4e3e-944d-e3b55b8e226c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:22.787478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5350df-c3e5-4e3e-944d-e3b55b8e226c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:22.788561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5350df-c3e5-4e3e-944d-e3b55b8e226c None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:22.789557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5350df-c3e5-4e3e-944d-e3b55b8e226c None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:22.791084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5350df-c3e5-4e3e-944d-e3b55b8e226c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:22.797088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 45/89] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:22 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:22.797316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5350df-c3e5-4e3e-944d-e3b55b8e226c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:22.798293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:22.830015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:22.863299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4124c1-d856-43c1-a30a-3e51147a806f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:22.903364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24299aa6-3a4a-4046-9141-c44d9d6d1929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:22.904305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24299aa6-3a4a-4046-9141-c44d9d6d1929 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:22.904807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24299aa6-3a4a-4046-9141-c44d9d6d1929 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:22.905188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24299aa6-3a4a-4046-9141-c44d9d6d1929 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:22.906433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24299aa6-3a4a-4046-9141-c44d9d6d1929 None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:22.908005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24299aa6-3a4a-4046-9141-c44d9d6d1929 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:22.910217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 45/90] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:22 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:22.911199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24299aa6-3a4a-4046-9141-c44d9d6d1929 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:22.928488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5350df-c3e5-4e3e-944d-e3b55b8e226c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:22.974860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bfda4-2115-436c-8ab3-a41d0017ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:22.976474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bfda4-2115-436c-8ab3-a41d0017ecf9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:22.977397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bfda4-2115-436c-8ab3-a41d0017ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:22.977857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bfda4-2115-436c-8ab3-a41d0017ecf9 None admin] PATH_INFO: `/v3/users/1a50181ffd4545db88d27e0331cd3f67` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:22.979910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4bfda4-2115-436c-8ab3-a41d0017ecf9 None admin] RBAC: Authorizing `identity:get_user(user_id=1a50181ffd4545db88d27e0331cd3f67)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:22.981607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4bfda4-2115-436c-8ab3-a41d0017ecf9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:22.984238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 46/91] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:22 2022] GET /identity/v3/users/1a50181ffd4545db88d27e0331cd3f67 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:22.985175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bfda4-2115-436c-8ab3-a41d0017ecf9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:23.028009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24299aa6-3a4a-4046-9141-c44d9d6d1929 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:23.048758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:23.063747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cab9a-b600-4c1e-bf8e-8841cf9a0f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:23.064525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cab9a-b600-4c1e-bf8e-8841cf9a0f53 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:23.064919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cab9a-b600-4c1e-bf8e-8841cf9a0f53 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:23.065291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14cab9a-b600-4c1e-bf8e-8841cf9a0f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:23.066144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14cab9a-b600-4c1e-bf8e-8841cf9a0f53 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=af2562b4c9384f15b1d6a92b56581f77, user.id=6789433c4ccf40c89d963e435a40ba65) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:23.066652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14cab9a-b600-4c1e-bf8e-8841cf9a0f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:23.068559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14cab9a-b600-4c1e-bf8e-8841cf9a0f53 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:23.075017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 46/92] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:22 2022] GET /identity/v3/role_assignments?user.id=6789433c4ccf40c89d963e435a40ba65&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:23.076009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cab9a-b600-4c1e-bf8e-8841cf9a0f53 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:23.119507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:23.138495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bfda4-2115-436c-8ab3-a41d0017ecf9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:23.212678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586ccff-fb36-406e-8ee5-07d3a8fb1cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:23.213312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586ccff-fb36-406e-8ee5-07d3a8fb1cfd None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:23.213591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586ccff-fb36-406e-8ee5-07d3a8fb1cfd None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:23.213866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586ccff-fb36-406e-8ee5-07d3a8fb1cfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:23.214625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586ccff-fb36-406e-8ee5-07d3a8fb1cfd None admin] RBAC: Adding query filter params (role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=af2562b4c9384f15b1d6a92b56581f77, user.id=1a50181ffd4545db88d27e0331cd3f67) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:23.214940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586ccff-fb36-406e-8ee5-07d3a8fb1cfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:23.216522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586ccff-fb36-406e-8ee5-07d3a8fb1cfd None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:23.223200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 47/93] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:22 2022] GET /identity/v3/role_assignments?user.id=1a50181ffd4545db88d27e0331cd3f67&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:23.233749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586ccff-fb36-406e-8ee5-07d3a8fb1cfd None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:23.313927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14cab9a-b600-4c1e-bf8e-8841cf9a0f53 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:23.389730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9419263-913e-4d91-845a-ef8bf84dfb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:23.390351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9419263-913e-4d91-845a-ef8bf84dfb86 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:23.390635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9419263-913e-4d91-845a-ef8bf84dfb86 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:23.390974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9419263-913e-4d91-845a-ef8bf84dfb86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:23.391646 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9419263-913e-4d91-845a-ef8bf84dfb86 None admin] RBAC: Adding query filter params (role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=af2562b4c9384f15b1d6a92b56581f77, user.id=1a50181ffd4545db88d27e0331cd3f67) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:23.391956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9419263-913e-4d91-845a-ef8bf84dfb86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:23.395146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9419263-913e-4d91-845a-ef8bf84dfb86 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:23.410569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 47/94] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:23 2022] GET /identity/v3/role_assignments?user.id=1a50181ffd4545db88d27e0331cd3f67&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:23.456788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586ccff-fb36-406e-8ee5-07d3a8fb1cfd None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:23.534824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58af12-9006-4f89-ba80-05ac1dbbdfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:23.535383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58af12-9006-4f89-ba80-05ac1dbbdfac None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:23.535666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58af12-9006-4f89-ba80-05ac1dbbdfac None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:23.535939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58af12-9006-4f89-ba80-05ac1dbbdfac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:23.566589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:23.642786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58af12-9006-4f89-ba80-05ac1dbbdfac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:23.644287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58af12-9006-4f89-ba80-05ac1dbbdfac None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:23.759038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 48/95] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:23.764910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9419263-913e-4d91-845a-ef8bf84dfb86 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:23.838832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:23.946061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9419263-913e-4d91-845a-ef8bf84dfb86 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:23.993451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fada7ca-c4a5-4fca-8184-0867b0c9ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:23.994103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada7ca-c4a5-4fca-8184-0867b0c9ad45 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:23.994954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada7ca-c4a5-4fca-8184-0867b0c9ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:23.995247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada7ca-c4a5-4fca-8184-0867b0c9ad45 None admin] PATH_INFO: `/v3/projects/d08c7d83dc8447d0bfeb9920ad510b61` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:23.996390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fada7ca-c4a5-4fca-8184-0867b0c9ad45 None admin] RBAC: Authorizing `identity:get_project(project_id=d08c7d83dc8447d0bfeb9920ad510b61)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:23.998133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fada7ca-c4a5-4fca-8184-0867b0c9ad45 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:24.000589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 48/96] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:23 2022] GET /identity/v3/projects/d08c7d83dc8447d0bfeb9920ad510b61 => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:24.011663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58af12-9006-4f89-ba80-05ac1dbbdfac None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:24.092617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:24.154318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58af12-9006-4f89-ba80-05ac1dbbdfac None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:24.196221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a1e96-c65b-4044-ae10-099fba562fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:24.196854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a1e96-c65b-4044-ae10-099fba562fee None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:24.197136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a1e96-c65b-4044-ae10-099fba562fee None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:24.197408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a1e96-c65b-4044-ae10-099fba562fee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:24.198140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a1e96-c65b-4044-ae10-099fba562fee None admin] RBAC: Adding query filter params (role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=d08c7d83dc8447d0bfeb9920ad510b61, user.id=1a50181ffd4545db88d27e0331cd3f67) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:24.198448 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a1e96-c65b-4044-ae10-099fba562fee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:24.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a1e96-c65b-4044-ae10-099fba562fee None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:24.205716 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 49/97] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:24 2022] GET /identity/v3/role_assignments?user.id=1a50181ffd4545db88d27e0331cd3f67&scope.project.id=d08c7d83dc8447d0bfeb9920ad510b61&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:25.286638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ea19-d20f-491d-9ea2-eed374e52e78 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:25.287140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ea19-d20f-491d-9ea2-eed374e52e78 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:25.287236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687ea19-d20f-491d-9ea2-eed374e52e78 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:25.288741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 49/98] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:25.300915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362fd3d-b938-42c5-96f4-8e6df0d519d6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:25.301342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362fd3d-b938-42c5-96f4-8e6df0d519d6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:25.301516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362fd3d-b938-42c5-96f4-8e6df0d519d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:25.325195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b362fd3d-b938-42c5-96f4-8e6df0d519d6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:25.326387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b362fd3d-b938-42c5-96f4-8e6df0d519d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:25.354363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c046241-0cae-4856-976c-0d892645b007 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:25.354679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c046241-0cae-4856-976c-0d892645b007 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:25.354916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c046241-0cae-4856-976c-0d892645b007 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:25.356743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 50/99] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:25.361520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd538e18-2a86-47e9-80ed-b37912a199e0 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:25.361808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd538e18-2a86-47e9-80ed-b37912a199e0 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:25.362086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd538e18-2a86-47e9-80ed-b37912a199e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:25.389316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-fd538e18-2a86-47e9-80ed-b37912a199e0 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:25.392378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fd538e18-2a86-47e9-80ed-b37912a199e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:25.487518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 50/100] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:25.536282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 51/101] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:25.909354 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5e976-3a8d-4e8f-9a0c-beb05fdc92ee None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:25.909742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5e976-3a8d-4e8f-9a0c-beb05fdc92ee None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:25.909921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5e976-3a8d-4e8f-9a0c-beb05fdc92ee None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:25.911507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 51/102] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:25.916534 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646be417-e196-4c57-a75d-668c65858b53 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:25.916829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646be417-e196-4c57-a75d-668c65858b53 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:25.917106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646be417-e196-4c57-a75d-668c65858b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:25.940587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5e976-3a8d-4e8f-9a0c-beb05fdc92ee None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:25.943704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-646be417-e196-4c57-a75d-668c65858b53 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:25.945565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-646be417-e196-4c57-a75d-668c65858b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:26.001516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:26.047483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:26.048942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 52/103] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:26.049803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646be417-e196-4c57-a75d-668c65858b53 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:26.114942 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5e976-3a8d-4e8f-9a0c-beb05fdc92ee None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:26.183609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b4fd3-5a1c-40eb-b9a9-f5d16a2af12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:26.184607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b4fd3-5a1c-40eb-b9a9-f5d16a2af12b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.185090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b4fd3-5a1c-40eb-b9a9-f5d16a2af12b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.185559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b4fd3-5a1c-40eb-b9a9-f5d16a2af12b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.191141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b4fd3-5a1c-40eb-b9a9-f5d16a2af12b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:26.192904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b4fd3-5a1c-40eb-b9a9-f5d16a2af12b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:26.195610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 52/104] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:26.200817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b4fd3-5a1c-40eb-b9a9-f5d16a2af12b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:26.251457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646be417-e196-4c57-a75d-668c65858b53 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:26.327437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfb90a-a5c3-4755-88eb-9d898fa34288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:26.329813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfb90a-a5c3-4755-88eb-9d898fa34288 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.331460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfb90a-a5c3-4755-88eb-9d898fa34288 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.331942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfb90a-a5c3-4755-88eb-9d898fa34288 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.357368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dfb90a-a5c3-4755-88eb-9d898fa34288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-93dfb90a-a5c3-4755-88eb-9d898fa34288 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:26.369576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:26.370447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:26.371213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:26.373603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 53/105] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:26.377189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7a6d2-9cac-45bf-a05c-99bc1895ff68 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.380415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7a6d2-9cac-45bf-a05c-99bc1895ff68 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.380938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7a6d2-9cac-45bf-a05c-99bc1895ff68 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.386519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 54/106] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:26.387494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7a6d2-9cac-45bf-a05c-99bc1895ff68 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:26.422268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b4fd3-5a1c-40eb-b9a9-f5d16a2af12b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:26.447480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:26.454924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:26.474298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44e0cf-5dad-44e1-9812-898fbccb4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:26.475175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44e0cf-5dad-44e1-9812-898fbccb4cf7 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.475882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44e0cf-5dad-44e1-9812-898fbccb4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.476306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44e0cf-5dad-44e1-9812-898fbccb4cf7 None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.477422 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44e0cf-5dad-44e1-9812-898fbccb4cf7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:26.479906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44e0cf-5dad-44e1-9812-898fbccb4cf7 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:26.520666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 53/107] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:16:26 2022] POST /identity/v3/projects => generated 302 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:16:26.523895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d50ce1-aa0b-443b-b7fa-e274f6034a9d None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.524361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d50ce1-aa0b-443b-b7fa-e274f6034a9d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.524981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d50ce1-aa0b-443b-b7fa-e274f6034a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.546704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7a6d2-9cac-45bf-a05c-99bc1895ff68 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:26.549046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-57d50ce1-aa0b-443b-b7fa-e274f6034a9d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:26.551597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-57d50ce1-aa0b-443b-b7fa-e274f6034a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:26.590236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d4cc5-1a6d-4b7b-a4e2-1902a36867fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:26.591013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d4cc5-1a6d-4b7b-a4e2-1902a36867fb None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.591398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d4cc5-1a6d-4b7b-a4e2-1902a36867fb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.591804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d4cc5-1a6d-4b7b-a4e2-1902a36867fb None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.592827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d4cc5-1a6d-4b7b-a4e2-1902a36867fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:26.593274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d4cc5-1a6d-4b7b-a4e2-1902a36867fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:26.594932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d4cc5-1a6d-4b7b-a4e2-1902a36867fb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:26.606925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 55/108] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:26.612570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d4cc5-1a6d-4b7b-a4e2-1902a36867fb None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:26.645196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 54/109] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:26.646292 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d50ce1-aa0b-443b-b7fa-e274f6034a9d None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:26.786408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:26.787642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:26.796137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d4cc5-1a6d-4b7b-a4e2-1902a36867fb None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:26.796528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d50ce1-aa0b-443b-b7fa-e274f6034a9d None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:26.837381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf74db-8b0a-43aa-a177-0fa16cf100d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:26.837580 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf74db-8b0a-43aa-a177-0fa16cf100d8 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.837823 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf74db-8b0a-43aa-a177-0fa16cf100d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.838085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf74db-8b0a-43aa-a177-0fa16cf100d8 None admin] PATH_INFO: `/v3/users/ee799e01b80643ab827342efcfc1c01e` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.845352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4579c32-7e4a-40b9-acb1-deda7d9cc9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:26.845352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579c32-7e4a-40b9-acb1-deda7d9cc9ee None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.845579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579c32-7e4a-40b9-acb1-deda7d9cc9ee None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.845783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4579c32-7e4a-40b9-acb1-deda7d9cc9ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.858207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4579c32-7e4a-40b9-acb1-deda7d9cc9ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:26.860004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf74db-8b0a-43aa-a177-0fa16cf100d8 None admin] RBAC: Authorizing `identity:get_user(user_id=ee799e01b80643ab827342efcfc1c01e)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:26.861586 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf74db-8b0a-43aa-a177-0fa16cf100d8 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:26.863517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 55/110] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:26 2022] GET /identity/v3/users/ee799e01b80643ab827342efcfc1c01e => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-d4579c32-7e4a-40b9-acb1-deda7d9cc9ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:26.863619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:26.864397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:26.865190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:26.866294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b7034-c212-428c-bee5-d98cc86c00d6 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.866370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 56/111] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:16:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:26.866435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b7034-c212-428c-bee5-d98cc86c00d6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.866435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b7034-c212-428c-bee5-d98cc86c00d6 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.866557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd05111-dacf-4a49-87ab-9087236df2f3 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.867239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd05111-dacf-4a49-87ab-9087236df2f3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.867336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 56/112] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:26.867404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd05111-dacf-4a49-87ab-9087236df2f3 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.871744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 57/113] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:26.872734 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b7034-c212-428c-bee5-d98cc86c00d6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:26.884518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c42cc6-0775-4d23-b93a-8aee16924278 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:26.884867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c42cc6-0775-4d23-b93a-8aee16924278 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:26.885146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c42cc6-0775-4d23-b93a-8aee16924278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:26.908971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e7c42cc6-0775-4d23-b93a-8aee16924278 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:26.910479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e7c42cc6-0775-4d23-b93a-8aee16924278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:27.000663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 58/114] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:27.002005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01d58-793f-4cf4-9830-a4d844ad31a7 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.002292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01d58-793f-4cf4-9830-a4d844ad31a7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.002619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01d58-793f-4cf4-9830-a4d844ad31a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.011407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b7034-c212-428c-bee5-d98cc86c00d6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:27.021622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-60f01d58-793f-4cf4-9830-a4d844ad31a7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:27.023531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-60f01d58-793f-4cf4-9830-a4d844ad31a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:27.050309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8525bf-05e1-48a8-a784-48acc36f1ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:27.051329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8525bf-05e1-48a8-a784-48acc36f1ff0 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.051836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8525bf-05e1-48a8-a784-48acc36f1ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.052354 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8525bf-05e1-48a8-a784-48acc36f1ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.053802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8525bf-05e1-48a8-a784-48acc36f1ff0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:27.054350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8525bf-05e1-48a8-a784-48acc36f1ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:27.056190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8525bf-05e1-48a8-a784-48acc36f1ff0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:27.065747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 57/115] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:27.071503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8525bf-05e1-48a8-a784-48acc36f1ff0 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.093813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:27.097392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 59/116] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:27.099336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f01d58-793f-4cf4-9830-a4d844ad31a7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.116346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:27.210980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f01d58-793f-4cf4-9830-a4d844ad31a7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:27.259330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15fddf-11b4-4f2f-a276-dd4ece0f2630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:27.259982 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fddf-11b4-4f2f-a276-dd4ece0f2630 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.260268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fddf-11b4-4f2f-a276-dd4ece0f2630 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.266184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fddf-11b4-4f2f-a276-dd4ece0f2630 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.266294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8525bf-05e1-48a8-a784-48acc36f1ff0 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:27.284489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15fddf-11b4-4f2f-a276-dd4ece0f2630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-7e15fddf-11b4-4f2f-a276-dd4ece0f2630 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:27.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:27.291159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:27.291974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 60/117] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:27.296055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15fddf-11b4-4f2f-a276-dd4ece0f2630 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.325363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd15ce3-00dd-467c-bbb6-2ee0b2e1f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:27.326250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd15ce3-00dd-467c-bbb6-2ee0b2e1f990 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.326597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd15ce3-00dd-467c-bbb6-2ee0b2e1f990 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.326867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd15ce3-00dd-467c-bbb6-2ee0b2e1f990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.383552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:27.415774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15fddf-11b4-4f2f-a276-dd4ece0f2630 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:27.422308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd15ce3-00dd-467c-bbb6-2ee0b2e1f990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:27.424467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd15ce3-00dd-467c-bbb6-2ee0b2e1f990 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:27.444819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:27.456754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c330f-7c01-4d07-aadc-ef78625fe3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:27.457391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c330f-7c01-4d07-aadc-ef78625fe3a7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.457672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c330f-7c01-4d07-aadc-ef78625fe3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.457939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c330f-7c01-4d07-aadc-ef78625fe3a7 None admin] PATH_INFO: `/v3/users/6789433c4ccf40c89d963e435a40ba65` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.459088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c330f-7c01-4d07-aadc-ef78625fe3a7 None admin] RBAC: Authorizing `identity:get_user(user_id=6789433c4ccf40c89d963e435a40ba65)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:27.461255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c330f-7c01-4d07-aadc-ef78625fe3a7 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:27.463336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 61/118] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/users/6789433c4ccf40c89d963e435a40ba65 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:27.463607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c330f-7c01-4d07-aadc-ef78625fe3a7 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.533805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 58/119] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:27.534475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd15ce3-00dd-467c-bbb6-2ee0b2e1f990 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.572340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c330f-7c01-4d07-aadc-ef78625fe3a7 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:27.607291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ed63f-3ddf-4bfc-94af-6e98fde86724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:27.608181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ed63f-3ddf-4bfc-94af-6e98fde86724 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.608707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ed63f-3ddf-4bfc-94af-6e98fde86724 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.609189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ed63f-3ddf-4bfc-94af-6e98fde86724 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.610054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ed63f-3ddf-4bfc-94af-6e98fde86724 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:27.610445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ed63f-3ddf-4bfc-94af-6e98fde86724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:27.612127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ed63f-3ddf-4bfc-94af-6e98fde86724 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:27.620797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 62/120] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:27.621760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ed63f-3ddf-4bfc-94af-6e98fde86724 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.626002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:27.642401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd15ce3-00dd-467c-bbb6-2ee0b2e1f990 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:27.661884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:27.681070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7c94a-9add-4694-a055-7ce6a1a9d7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:27.681706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7c94a-9add-4694-a055-7ce6a1a9d7dd None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.681999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7c94a-9add-4694-a055-7ce6a1a9d7dd None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.682251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7c94a-9add-4694-a055-7ce6a1a9d7dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.693149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7c94a-9add-4694-a055-7ce6a1a9d7dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-89a7c94a-9add-4694-a055-7ce6a1a9d7dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:27.698285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:27.699150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:27.699884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 59/121] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:27.700538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7c94a-9add-4694-a055-7ce6a1a9d7dd None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.730607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ed63f-3ddf-4bfc-94af-6e98fde86724 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:27.769860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0864ac9-8f26-4724-9490-a5212c7dffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:27.770499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0864ac9-8f26-4724-9490-a5212c7dffd7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.770780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0864ac9-8f26-4724-9490-a5212c7dffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.771056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0864ac9-8f26-4724-9490-a5212c7dffd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.782101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0864ac9-8f26-4724-9490-a5212c7dffd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f0864ac9-8f26-4724-9490-a5212c7dffd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:27.787352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:27.789079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:27.789851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:27.793892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 63/122] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:27.794704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0864ac9-8f26-4724-9490-a5212c7dffd7 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.815874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7c94a-9add-4694-a055-7ce6a1a9d7dd None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:27.855278 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b720e4-9062-45bd-a511-abebaf9efd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:27.855923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b720e4-9062-45bd-a511-abebaf9efd1e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.856204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b720e4-9062-45bd-a511-abebaf9efd1e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.856475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b720e4-9062-45bd-a511-abebaf9efd1e None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.857709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b720e4-9062-45bd-a511-abebaf9efd1e None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:27.859126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b720e4-9062-45bd-a511-abebaf9efd1e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:27.860806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 60/123] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:27.861570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b720e4-9062-45bd-a511-abebaf9efd1e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.871820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:27.912346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:27.918162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0864ac9-8f26-4724-9490-a5212c7dffd7 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:27.955979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f2f96-ea61-4ec7-ac9d-92aa89f2a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:27.956965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f2f96-ea61-4ec7-ac9d-92aa89f2a04d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:27.957441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f2f96-ea61-4ec7-ac9d-92aa89f2a04d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:27.957904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f2f96-ea61-4ec7-ac9d-92aa89f2a04d None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:27.959237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f2f96-ea61-4ec7-ac9d-92aa89f2a04d None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:27.961490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f2f96-ea61-4ec7-ac9d-92aa89f2a04d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:27.963312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 64/124] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:27.964263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f2f96-ea61-4ec7-ac9d-92aa89f2a04d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:27.977036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b720e4-9062-45bd-a511-abebaf9efd1e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.011813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712849b-20bd-479d-9870-b0cfbe673145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.012448 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712849b-20bd-479d-9870-b0cfbe673145 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.012770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712849b-20bd-479d-9870-b0cfbe673145 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.013050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712849b-20bd-479d-9870-b0cfbe673145 None admin] PATH_INFO: `/v3/users/1a50181ffd4545db88d27e0331cd3f67` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.014196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712849b-20bd-479d-9870-b0cfbe673145 None admin] RBAC: Authorizing `identity:get_user(user_id=1a50181ffd4545db88d27e0331cd3f67)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.015665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712849b-20bd-479d-9870-b0cfbe673145 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.021645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 61/125] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/users/1a50181ffd4545db88d27e0331cd3f67 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:28.021645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712849b-20bd-479d-9870-b0cfbe673145 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:28.077685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f2f96-ea61-4ec7-ac9d-92aa89f2a04d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.109040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:28.115258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc6b0d-b936-4f7c-a995-0db9700d6a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.116275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc6b0d-b936-4f7c-a995-0db9700d6a67 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.117091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc6b0d-b936-4f7c-a995-0db9700d6a67 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.117472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc6b0d-b936-4f7c-a995-0db9700d6a67 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.118481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc6b0d-b936-4f7c-a995-0db9700d6a67 None admin] RBAC: Adding query filter params (name=member) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:28.119126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc6b0d-b936-4f7c-a995-0db9700d6a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.121362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc6b0d-b936-4f7c-a995-0db9700d6a67 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.122844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712849b-20bd-479d-9870-b0cfbe673145 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.132123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 65/126] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:28.133344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc6b0d-b936-4f7c-a995-0db9700d6a67 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:28.143763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:28.167991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8fb5a-c182-481a-af04-890a4db5f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.168636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8fb5a-c182-481a-af04-890a4db5f9ff None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.168917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8fb5a-c182-481a-af04-890a4db5f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.169193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8fb5a-c182-481a-af04-890a4db5f9ff None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.170061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8fb5a-c182-481a-af04-890a4db5f9ff None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:28.170171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8fb5a-c182-481a-af04-890a4db5f9ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.171552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8fb5a-c182-481a-af04-890a4db5f9ff None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.180154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 62/127] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:28.181662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8fb5a-c182-481a-af04-890a4db5f9ff None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:28.244599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc6b0d-b936-4f7c-a995-0db9700d6a67 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.281003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525dd49-99aa-43eb-a8e8-8ad118e0a4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.281670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525dd49-99aa-43eb-a8e8-8ad118e0a4b4 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.281953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525dd49-99aa-43eb-a8e8-8ad118e0a4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.282229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525dd49-99aa-43eb-a8e8-8ad118e0a4b4 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77/users/6789433c4ccf40c89d963e435a40ba65/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.284154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525dd49-99aa-43eb-a8e8-8ad118e0a4b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=af2562b4c9384f15b1d6a92b56581f77, user_id=6789433c4ccf40c89d963e435a40ba65, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.285658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525dd49-99aa-43eb-a8e8-8ad118e0a4b4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.295835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8fb5a-c182-481a-af04-890a4db5f9ff None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.299136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 66/128] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:16:28 2022] PUT /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77/users/6789433c4ccf40c89d963e435a40ba65/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:16:28.300182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525dd49-99aa-43eb-a8e8-8ad118e0a4b4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:28.343490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4797d9-5e24-489a-8e49-62b450064bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.344143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4797d9-5e24-489a-8e49-62b450064bc9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.344426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4797d9-5e24-489a-8e49-62b450064bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.344709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4797d9-5e24-489a-8e49-62b450064bc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.345461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4797d9-5e24-489a-8e49-62b450064bc9 None admin] RBAC: Adding query filter params (role.id=a6588e7acf534502a191a0bca0d46c39, scope.project.id=af2562b4c9384f15b1d6a92b56581f77, user.id=ee799e01b80643ab827342efcfc1c01e) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:28.345773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4797d9-5e24-489a-8e49-62b450064bc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.347298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4797d9-5e24-489a-8e49-62b450064bc9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.372602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 63/129] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:28 2022] GET /identity/v3/role_assignments?user.id=ee799e01b80643ab827342efcfc1c01e&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=a6588e7acf534502a191a0bca0d46c39 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:28.372602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4797d9-5e24-489a-8e49-62b450064bc9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:28.401806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:28.465787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:28.504448 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525dd49-99aa-43eb-a8e8-8ad118e0a4b4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.545228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4797d9-5e24-489a-8e49-62b450064bc9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.564904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c4d9-c663-47c0-a79b-87072cd8e7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.565696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c4d9-c663-47c0-a79b-87072cd8e7eb None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.566076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c4d9-c663-47c0-a79b-87072cd8e7eb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.566445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c4d9-c663-47c0-a79b-87072cd8e7eb None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.567680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0c4d9-c663-47c0-a79b-87072cd8e7eb None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.569462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0c4d9-c663-47c0-a79b-87072cd8e7eb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.571471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 67/130] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:28 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:28.572177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c4d9-c663-47c0-a79b-87072cd8e7eb None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:28.584209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8e510-4888-4a0b-88d3-cb2582ae0e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.585036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8e510-4888-4a0b-88d3-cb2582ae0e6c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.585541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8e510-4888-4a0b-88d3-cb2582ae0e6c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.585817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8e510-4888-4a0b-88d3-cb2582ae0e6c None admin] PATH_INFO: `/v3/users/1a50181ffd4545db88d27e0331cd3f67` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.587462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8e510-4888-4a0b-88d3-cb2582ae0e6c None admin] RBAC: Authorizing `identity:get_user(user_id=1a50181ffd4545db88d27e0331cd3f67)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.589344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8e510-4888-4a0b-88d3-cb2582ae0e6c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.591558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 64/131] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:28 2022] GET /identity/v3/users/1a50181ffd4545db88d27e0331cd3f67 => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:28.592316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8e510-4888-4a0b-88d3-cb2582ae0e6c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:28.706176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c4d9-c663-47c0-a79b-87072cd8e7eb None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.733059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:28.748056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17422ae-2f36-4d00-956b-661f6150474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.748769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17422ae-2f36-4d00-956b-661f6150474f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.749052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17422ae-2f36-4d00-956b-661f6150474f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.749362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17422ae-2f36-4d00-956b-661f6150474f None admin] PATH_INFO: `/v3/users/1a50181ffd4545db88d27e0331cd3f67` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.751136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17422ae-2f36-4d00-956b-661f6150474f None admin] RBAC: Authorizing `identity:get_user(user_id=1a50181ffd4545db88d27e0331cd3f67)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.752958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17422ae-2f36-4d00-956b-661f6150474f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.755280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 68/132] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:28 2022] GET /identity/v3/users/1a50181ffd4545db88d27e0331cd3f67 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:28.756059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17422ae-2f36-4d00-956b-661f6150474f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:28.781247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8e510-4888-4a0b-88d3-cb2582ae0e6c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.791923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:28.825368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59a8e6-919f-423b-9767-e6ce2d79cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.826515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a8e6-919f-423b-9767-e6ce2d79cfd0 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.826931 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a8e6-919f-423b-9767-e6ce2d79cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.827465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59a8e6-919f-423b-9767-e6ce2d79cfd0 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77/users/1a50181ffd4545db88d27e0331cd3f67/roles/2986c98e353a489592c69ada5c3c2aaf` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.830640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a8e6-919f-423b-9767-e6ce2d79cfd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=af2562b4c9384f15b1d6a92b56581f77, user_id=1a50181ffd4545db88d27e0331cd3f67, role_id=2986c98e353a489592c69ada5c3c2aaf)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.832831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59a8e6-919f-423b-9767-e6ce2d79cfd0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.856639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 65/133] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:16:28 2022] PUT /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77/users/1a50181ffd4545db88d27e0331cd3f67/roles/2986c98e353a489592c69ada5c3c2aaf => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:16:28.858815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c1ff2-174b-4403-9f42-e6220381b1f7 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.859249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c1ff2-174b-4403-9f42-e6220381b1f7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.859689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c1ff2-174b-4403-9f42-e6220381b1f7 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.862434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 66/134] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:28.863721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c1ff2-174b-4403-9f42-e6220381b1f7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:28.884074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17422ae-2f36-4d00-956b-661f6150474f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:28.926815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e6326-ed7f-488f-8e8c-09ec88c22477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:28.927557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e6326-ed7f-488f-8e8c-09ec88c22477 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.927891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e6326-ed7f-488f-8e8c-09ec88c22477 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.928770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e6326-ed7f-488f-8e8c-09ec88c22477 None admin] PATH_INFO: `/v3/projects/d08c7d83dc8447d0bfeb9920ad510b61` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.930045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e6326-ed7f-488f-8e8c-09ec88c22477 None admin] RBAC: Authorizing `identity:get_project(project_id=d08c7d83dc8447d0bfeb9920ad510b61)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:28.931700 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e6326-ed7f-488f-8e8c-09ec88c22477 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:28.933921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 69/135] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:28 2022] GET /identity/v3/projects/d08c7d83dc8447d0bfeb9920ad510b61 => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:28.935318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d64f33-4244-4ca4-ab64-b2240df75ccc None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:28.935656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d64f33-4244-4ca4-ab64-b2240df75ccc None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:28.935978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d64f33-4244-4ca4-ab64-b2240df75ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:28.956909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-07d64f33-4244-4ca4-ab64-b2240df75ccc None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:28.958136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-07d64f33-4244-4ca4-ab64-b2240df75ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:29.006970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c1ff2-174b-4403-9f42-e6220381b1f7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:29.026519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:29.042860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 70/136] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:29.043920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d64f33-4244-4ca4-ab64-b2240df75ccc None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:29.058290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813dce9-f574-4479-906e-e9731742d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:29.059177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813dce9-f574-4479-906e-e9731742d523 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:29.059596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813dce9-f574-4479-906e-e9731742d523 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:29.060006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813dce9-f574-4479-906e-e9731742d523 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:29.061376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813dce9-f574-4479-906e-e9731742d523 None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:29.063640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813dce9-f574-4479-906e-e9731742d523 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:29.065757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 67/137] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:28 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:29.071776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813dce9-f574-4479-906e-e9731742d523 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:29.119588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:29.183202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d64f33-4244-4ca4-ab64-b2240df75ccc None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:29.200167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813dce9-f574-4479-906e-e9731742d523 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:29.239024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0cdfb-87ef-47be-85b4-24cfc9658993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:29.241334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0cdfb-87ef-47be-85b4-24cfc9658993 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:29.242804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0cdfb-87ef-47be-85b4-24cfc9658993 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:29.243312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0cdfb-87ef-47be-85b4-24cfc9658993 None admin] PATH_INFO: `/v3/projects/d08c7d83dc8447d0bfeb9920ad510b61/users/1a50181ffd4545db88d27e0331cd3f67/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:29.247264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0cdfb-87ef-47be-85b4-24cfc9658993 None admin] RBAC: Authorizing `identity:create_grant(project_id=d08c7d83dc8447d0bfeb9920ad510b61, user_id=1a50181ffd4545db88d27e0331cd3f67, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:29.255035 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0cdfb-87ef-47be-85b4-24cfc9658993 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:29.265212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e5089-0bb7-4b10-bca6-1f4e13a55fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:29.266061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e5089-0bb7-4b10-bca6-1f4e13a55fe5 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:29.266579 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e5089-0bb7-4b10-bca6-1f4e13a55fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:29.267109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e5089-0bb7-4b10-bca6-1f4e13a55fe5 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77/users/1a50181ffd4545db88d27e0331cd3f67/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:29.274212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e5089-0bb7-4b10-bca6-1f4e13a55fe5 None admin] RBAC: Authorizing `identity:create_grant(project_id=af2562b4c9384f15b1d6a92b56581f77, user_id=1a50181ffd4545db88d27e0331cd3f67, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:29.278864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e5089-0bb7-4b10-bca6-1f4e13a55fe5 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:29.279935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 71/138] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:16:29 2022] PUT /identity/v3/projects/d08c7d83dc8447d0bfeb9920ad510b61/users/1a50181ffd4545db88d27e0331cd3f67/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:16:29.299639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 68/139] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:16:29 2022] PUT /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77/users/1a50181ffd4545db88d27e0331cd3f67/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:16:29.451036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0cdfb-87ef-47be-85b4-24cfc9658993 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:29.514202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:29.577895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0cdfb-87ef-47be-85b4-24cfc9658993 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:29.617372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbdd78-b579-4cd7-9397-9ddae0159c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:29.618024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbdd78-b579-4cd7-9397-9ddae0159c66 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:29.618299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbdd78-b579-4cd7-9397-9ddae0159c66 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:29.618568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbdd78-b579-4cd7-9397-9ddae0159c66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:29.619685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbdd78-b579-4cd7-9397-9ddae0159c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:29.621251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbdd78-b579-4cd7-9397-9ddae0159c66 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:29.623112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 72/140] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:29.628317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e5089-0bb7-4b10-bca6-1f4e13a55fe5 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:29.738743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:29.746684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e5089-0bb7-4b10-bca6-1f4e13a55fe5 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:29.784700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeb745-ae17-4f6c-b428-2d20dc95f743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:29.785233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeb745-ae17-4f6c-b428-2d20dc95f743 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:29.785513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeb745-ae17-4f6c-b428-2d20dc95f743 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:29.785786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeb745-ae17-4f6c-b428-2d20dc95f743 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:29.786452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeb745-ae17-4f6c-b428-2d20dc95f743 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:29.787993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeb745-ae17-4f6c-b428-2d20dc95f743 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:29.815786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 69/141] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:16:29 2022] POST /identity/v3/users => generated 283 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:30.711859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d89fe-1713-4f50-aebb-0c000f27a672 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:30.712368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d89fe-1713-4f50-aebb-0c000f27a672 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:30.712440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d89fe-1713-4f50-aebb-0c000f27a672 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:30.714085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 73/142] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:30.719887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc536c9a-658b-49a3-895c-59b3c17168d3 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:30.720350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc536c9a-658b-49a3-895c-59b3c17168d3 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:30.720561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc536c9a-658b-49a3-895c-59b3c17168d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:30.745924 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-dc536c9a-658b-49a3-895c-59b3c17168d3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:30.747317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-dc536c9a-658b-49a3-895c-59b3c17168d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:30.845300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 70/143] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:31.215592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3387f62-020a-418b-a76d-16bb95b544a4 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:31.215951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3387f62-020a-418b-a76d-16bb95b544a4 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:31.216163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3387f62-020a-418b-a76d-16bb95b544a4 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:31.217842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 74/144] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:31.223720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffd213-6366-479e-b14d-089c2fd2569f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:31.224024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffd213-6366-479e-b14d-089c2fd2569f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:31.224307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdffd213-6366-479e-b14d-089c2fd2569f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:31.259827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fdffd213-6366-479e-b14d-089c2fd2569f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:31.261072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fdffd213-6366-479e-b14d-089c2fd2569f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:31.342955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 71/145] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:31.394268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3387f62-020a-418b-a76d-16bb95b544a4 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:31.518905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3387f62-020a-418b-a76d-16bb95b544a4 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:31.545226 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:31.567955 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b29e1-a321-479e-aaff-6b7e10bbe927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:31.568703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b29e1-a321-479e-aaff-6b7e10bbe927 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:31.569611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b29e1-a321-479e-aaff-6b7e10bbe927 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:31.569894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b29e1-a321-479e-aaff-6b7e10bbe927 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:31.583599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b29e1-a321-479e-aaff-6b7e10bbe927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-7c4b29e1-a321-479e-aaff-6b7e10bbe927 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:31.589359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:31.590221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:31.591423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 75/146] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:16:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:31.596524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffd213-6366-479e-b14d-089c2fd2569f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:31.617383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:31.729874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdffd213-6366-479e-b14d-089c2fd2569f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:31.758559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b29e1-a321-479e-aaff-6b7e10bbe927 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:31.812445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece02b1-9e71-41ea-99bc-bea457243e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:31.814344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece02b1-9e71-41ea-99bc-bea457243e7a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:31.814636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece02b1-9e71-41ea-99bc-bea457243e7a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:31.814906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece02b1-9e71-41ea-99bc-bea457243e7a None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:31.815583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece02b1-9e71-41ea-99bc-bea457243e7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:31.815881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece02b1-9e71-41ea-99bc-bea457243e7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:31.825493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece02b1-9e71-41ea-99bc-bea457243e7a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:31.840510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 72/147] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:31.846897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece02b1-9e71-41ea-99bc-bea457243e7a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:32.002217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:32.009632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b29e1-a321-479e-aaff-6b7e10bbe927 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:32.044854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece02b1-9e71-41ea-99bc-bea457243e7a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:32.065328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5dd47-31bc-4437-a21b-f423ba9af9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:32.065905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5dd47-31bc-4437-a21b-f423ba9af9f7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.066192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5dd47-31bc-4437-a21b-f423ba9af9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.066468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5dd47-31bc-4437-a21b-f423ba9af9f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.081748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5dd47-31bc-4437-a21b-f423ba9af9f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-a7b5dd47-31bc-4437-a21b-f423ba9af9f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:32.087259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:32.088125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:32.089308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 76/148] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:32.097524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5dd47-31bc-4437-a21b-f423ba9af9f7 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:32.144583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855364b7-28f9-4bc7-a6b2-4ca2a7c10b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:32.145239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855364b7-28f9-4bc7-a6b2-4ca2a7c10b40 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.145514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855364b7-28f9-4bc7-a6b2-4ca2a7c10b40 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.145784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855364b7-28f9-4bc7-a6b2-4ca2a7c10b40 None admin] PATH_INFO: `/v3/users/6789433c4ccf40c89d963e435a40ba65` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.148601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855364b7-28f9-4bc7-a6b2-4ca2a7c10b40 None admin] RBAC: Authorizing `identity:get_user(user_id=6789433c4ccf40c89d963e435a40ba65)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:32.148983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:32.150721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855364b7-28f9-4bc7-a6b2-4ca2a7c10b40 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:32.152643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 73/149] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:31 2022] GET /identity/v3/users/6789433c4ccf40c89d963e435a40ba65 => generated 247 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:32.158681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f367015-7784-4d8c-b231-56c0e3b1817a None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.158956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f367015-7784-4d8c-b231-56c0e3b1817a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.159226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f367015-7784-4d8c-b231-56c0e3b1817a None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.160717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 74/150] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:16:32.161241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f367015-7784-4d8c-b231-56c0e3b1817a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:32.282130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5dd47-31bc-4437-a21b-f423ba9af9f7 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:32.339703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f641-cae1-4595-b71b-66deb3cd995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:32.340606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f641-cae1-4595-b71b-66deb3cd995a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.340606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f641-cae1-4595-b71b-66deb3cd995a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.340852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f641-cae1-4595-b71b-66deb3cd995a None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.341479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38f641-cae1-4595-b71b-66deb3cd995a None admin] RBAC: Adding query filter params (name=reader) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:32.341781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38f641-cae1-4595-b71b-66deb3cd995a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:32.343185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38f641-cae1-4595-b71b-66deb3cd995a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:32.362841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 77/151] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:32.364044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c4b5-8fbe-4658-8270-0b664b113472 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.364326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c4b5-8fbe-4658-8270-0b664b113472 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.364626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87c4b5-8fbe-4658-8270-0b664b113472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.399307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5e87c4b5-8fbe-4658-8270-0b664b113472 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:32.400878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5e87c4b5-8fbe-4658-8270-0b664b113472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:32.421774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f367015-7784-4d8c-b231-56c0e3b1817a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:32.493480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71aad3-002d-4ed7-8d21-7d5476638789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:32.494097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71aad3-002d-4ed7-8d21-7d5476638789 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.494374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71aad3-002d-4ed7-8d21-7d5476638789 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.494696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71aad3-002d-4ed7-8d21-7d5476638789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.510563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:32.531290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 78/152] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:32.534144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc573f-78ee-4773-a265-5f93c508d661 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.534424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc573f-78ee-4773-a265-5f93c508d661 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.534692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc573f-78ee-4773-a265-5f93c508d661 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.540981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 79/153] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:32.541795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc573f-78ee-4773-a265-5f93c508d661 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:32.601446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:32.643650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71aad3-002d-4ed7-8d21-7d5476638789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:32.645252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71aad3-002d-4ed7-8d21-7d5476638789 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:32.706128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc573f-78ee-4773-a265-5f93c508d661 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:32.743858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f1b82-9dc1-47b2-a2e0-99ecb109c5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:32.744948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f1b82-9dc1-47b2-a2e0-99ecb109c5c0 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.745414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f1b82-9dc1-47b2-a2e0-99ecb109c5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.746004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f1b82-9dc1-47b2-a2e0-99ecb109c5c0 None admin] PATH_INFO: `/v3/users/ee799e01b80643ab827342efcfc1c01e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.747737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f1b82-9dc1-47b2-a2e0-99ecb109c5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=ee799e01b80643ab827342efcfc1c01e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:32.749643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f1b82-9dc1-47b2-a2e0-99ecb109c5c0 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:32.751768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 80/154] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:32 2022] GET /identity/v3/users/ee799e01b80643ab827342efcfc1c01e => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:32.753419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acf1f8-ccb9-40de-b065-3400b37ea696 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.753902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acf1f8-ccb9-40de-b065-3400b37ea696 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.754339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acf1f8-ccb9-40de-b065-3400b37ea696 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.755940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 81/155] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:32.757264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712505d-ac4e-44d4-9bd3-f35c63676f4d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.757834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712505d-ac4e-44d4-9bd3-f35c63676f4d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.758297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712505d-ac4e-44d4-9bd3-f35c63676f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.777422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-9712505d-ac4e-44d4-9bd3-f35c63676f4d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:32.779303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9712505d-ac4e-44d4-9bd3-f35c63676f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:32.818346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 75/156] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:32.819413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71aad3-002d-4ed7-8d21-7d5476638789 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:32.856406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 82/157] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:32.861118 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393859b2-e21d-4d74-980d-c9ff646d58d5 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:32.861624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393859b2-e21d-4d74-980d-c9ff646d58d5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:32.862079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393859b2-e21d-4d74-980d-c9ff646d58d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:32.904840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-393859b2-e21d-4d74-980d-c9ff646d58d5 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:32.907691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-393859b2-e21d-4d74-980d-c9ff646d58d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:32.908850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:32.944526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:33.020106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 83/158] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:33.023419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393859b2-e21d-4d74-980d-c9ff646d58d5 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:33.026023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71aad3-002d-4ed7-8d21-7d5476638789 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:33.104205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087ad21-d510-412b-ae2d-ddd9e747be39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:33.105150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087ad21-d510-412b-ae2d-ddd9e747be39 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:33.105602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087ad21-d510-412b-ae2d-ddd9e747be39 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:33.106100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087ad21-d510-412b-ae2d-ddd9e747be39 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:33.107789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087ad21-d510-412b-ae2d-ddd9e747be39 None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:33.109571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087ad21-d510-412b-ae2d-ddd9e747be39 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:33.111804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 76/159] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:32 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:33.112836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087ad21-d510-412b-ae2d-ddd9e747be39 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:33.247687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393859b2-e21d-4d74-980d-c9ff646d58d5 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:33.323707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bb08c-99ce-4ef3-90fa-35f1c249a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:33.324355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bb08c-99ce-4ef3-90fa-35f1c249a773 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:33.324665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bb08c-99ce-4ef3-90fa-35f1c249a773 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:33.324941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bb08c-99ce-4ef3-90fa-35f1c249a773 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:33.327191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885bb08c-99ce-4ef3-90fa-35f1c249a773 None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:33.328688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885bb08c-99ce-4ef3-90fa-35f1c249a773 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:33.336462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087ad21-d510-412b-ae2d-ddd9e747be39 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:33.340838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 84/160] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:33 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:33.341713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bb08c-99ce-4ef3-90fa-35f1c249a773 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:33.368933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:33.407848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:33.424703 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56243cc-4be1-4e20-b138-22586baf0feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:33.425356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56243cc-4be1-4e20-b138-22586baf0feb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:33.425635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56243cc-4be1-4e20-b138-22586baf0feb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:33.425905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56243cc-4be1-4e20-b138-22586baf0feb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:33.441173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56243cc-4be1-4e20-b138-22586baf0feb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-f56243cc-4be1-4e20-b138-22586baf0feb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:33.453265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:33.454216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:33.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 77/161] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:16:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:33.456039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56243cc-4be1-4e20-b138-22586baf0feb None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:33.524036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bb08c-99ce-4ef3-90fa-35f1c249a773 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:33.568572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8eff3-17f8-4267-a14c-f44da3920deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:33.569786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8eff3-17f8-4267-a14c-f44da3920deb None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:33.570069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8eff3-17f8-4267-a14c-f44da3920deb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:33.570343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8eff3-17f8-4267-a14c-f44da3920deb None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77/users/ee799e01b80643ab827342efcfc1c01e/roles/a6588e7acf534502a191a0bca0d46c39` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:33.577300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8eff3-17f8-4267-a14c-f44da3920deb None admin] RBAC: Authorizing `identity:create_grant(project_id=af2562b4c9384f15b1d6a92b56581f77, user_id=ee799e01b80643ab827342efcfc1c01e, role_id=a6588e7acf534502a191a0bca0d46c39)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:33.578864 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8eff3-17f8-4267-a14c-f44da3920deb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:33.604085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 85/162] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:16:33 2022] PUT /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77/users/ee799e01b80643ab827342efcfc1c01e/roles/a6588e7acf534502a191a0bca0d46c39 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:16:33.605099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8eff3-17f8-4267-a14c-f44da3920deb None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:33.723490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56243cc-4be1-4e20-b138-22586baf0feb None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:33.725256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:33.746848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8eff3-17f8-4267-a14c-f44da3920deb None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:33.791272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c8af9-c2f2-40e4-8a3f-cd935b8e3428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:33.791929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c8af9-c2f2-40e4-8a3f-cd935b8e3428 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:33.792178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c8af9-c2f2-40e4-8a3f-cd935b8e3428 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:33.795328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c8af9-c2f2-40e4-8a3f-cd935b8e3428 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:33.826689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c8af9-c2f2-40e4-8a3f-cd935b8e3428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-c25c8af9-c2f2-40e4-8a3f-cd935b8e3428 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:33.832251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:33.833258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:33.834316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 86/163] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:33.835607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c8af9-c2f2-40e4-8a3f-cd935b8e3428 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:33.857317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f9373-b058-40df-8b0a-622b430c6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:33.857985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f9373-b058-40df-8b0a-622b430c6234 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:33.858244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f9373-b058-40df-8b0a-622b430c6234 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:33.858513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f9373-b058-40df-8b0a-622b430c6234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:33.859259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f9373-b058-40df-8b0a-622b430c6234 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=af2562b4c9384f15b1d6a92b56581f77, user.id=6789433c4ccf40c89d963e435a40ba65) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:33.859566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f9373-b058-40df-8b0a-622b430c6234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:33.861092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f9373-b058-40df-8b0a-622b430c6234 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:33.883070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 78/164] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:33 2022] GET /identity/v3/role_assignments?user.id=6789433c4ccf40c89d963e435a40ba65&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 608 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:33.884077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f9373-b058-40df-8b0a-622b430c6234 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:33.962043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c8af9-c2f2-40e4-8a3f-cd935b8e3428 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:33.995606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:34.005664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5579fc6-2a16-4bf1-b5e0-5aaa23b97a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:34.007299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5579fc6-2a16-4bf1-b5e0-5aaa23b97a7f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.007694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5579fc6-2a16-4bf1-b5e0-5aaa23b97a7f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.007968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5579fc6-2a16-4bf1-b5e0-5aaa23b97a7f None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.008698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5579fc6-2a16-4bf1-b5e0-5aaa23b97a7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:34.009003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5579fc6-2a16-4bf1-b5e0-5aaa23b97a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:34.010543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5579fc6-2a16-4bf1-b5e0-5aaa23b97a7f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:34.020383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 87/165] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:16:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:34.025560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5579fc6-2a16-4bf1-b5e0-5aaa23b97a7f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:34.064911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:34.110225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f9373-b058-40df-8b0a-622b430c6234 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:34.147623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5579fc6-2a16-4bf1-b5e0-5aaa23b97a7f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:34.177339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066a5a1-8fd9-4bae-97ba-caeff395eeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:34.178203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066a5a1-8fd9-4bae-97ba-caeff395eeed None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.178483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066a5a1-8fd9-4bae-97ba-caeff395eeed None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.178821 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066a5a1-8fd9-4bae-97ba-caeff395eeed None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.189810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385e0ce-5047-46ab-8038-9604dddaddb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:34.190482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385e0ce-5047-46ab-8038-9604dddaddb6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.190764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385e0ce-5047-46ab-8038-9604dddaddb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.191151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385e0ce-5047-46ab-8038-9604dddaddb6 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.191825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385e0ce-5047-46ab-8038-9604dddaddb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:34.192125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385e0ce-5047-46ab-8038-9604dddaddb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:34.194947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385e0ce-5047-46ab-8038-9604dddaddb6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:34.204765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066a5a1-8fd9-4bae-97ba-caeff395eeed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:34.205544 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 88/166] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:34.206232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385e0ce-5047-46ab-8038-9604dddaddb6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-3066a5a1-8fd9-4bae-97ba-caeff395eeed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:34.209927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:34.210797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:34.211687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 79/167] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:34.216824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066a5a1-8fd9-4bae-97ba-caeff395eeed None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:34.296520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:34.337293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385e0ce-5047-46ab-8038-9604dddaddb6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:34.378230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440a77-f147-4fc4-8cb2-662f85715919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:34.378879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440a77-f147-4fc4-8cb2-662f85715919 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.379164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440a77-f147-4fc4-8cb2-662f85715919 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.379445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440a77-f147-4fc4-8cb2-662f85715919 None admin] PATH_INFO: `/v3/users/1a50181ffd4545db88d27e0331cd3f67` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.380648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95440a77-f147-4fc4-8cb2-662f85715919 None admin] RBAC: Authorizing `identity:get_user(user_id=1a50181ffd4545db88d27e0331cd3f67)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:34.383665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95440a77-f147-4fc4-8cb2-662f85715919 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:34.385339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 89/168] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:34 2022] GET /identity/v3/users/1a50181ffd4545db88d27e0331cd3f67 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:34.386132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440a77-f147-4fc4-8cb2-662f85715919 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:34.441635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:34.463613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066a5a1-8fd9-4bae-97ba-caeff395eeed None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:34.513383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440a77-f147-4fc4-8cb2-662f85715919 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:34.549916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f136dc6-fe06-42cc-8127-2cf80de536eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:34.550465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f136dc6-fe06-42cc-8127-2cf80de536eb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.550748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f136dc6-fe06-42cc-8127-2cf80de536eb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.551021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f136dc6-fe06-42cc-8127-2cf80de536eb None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.551706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f136dc6-fe06-42cc-8127-2cf80de536eb None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:34.552007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f136dc6-fe06-42cc-8127-2cf80de536eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:34.555222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd23c1d-b3c1-4aef-91f5-447dbe4c3d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:34.555938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd23c1d-b3c1-4aef-91f5-447dbe4c3d0a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.556218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd23c1d-b3c1-4aef-91f5-447dbe4c3d0a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.556508 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd23c1d-b3c1-4aef-91f5-447dbe4c3d0a None admin] PATH_INFO: `/v3/users/1a50181ffd4545db88d27e0331cd3f67` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.557537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f136dc6-fe06-42cc-8127-2cf80de536eb None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:34.557717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd23c1d-b3c1-4aef-91f5-447dbe4c3d0a None admin] RBAC: Authorizing `identity:get_user(user_id=1a50181ffd4545db88d27e0331cd3f67)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:34.559146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd23c1d-b3c1-4aef-91f5-447dbe4c3d0a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:34.561203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 90/169] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:34 2022] GET /identity/v3/users/1a50181ffd4545db88d27e0331cd3f67 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:34.562050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd23c1d-b3c1-4aef-91f5-447dbe4c3d0a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:34.570186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 80/170] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:34.571599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095338d-cb51-46ff-b958-8da207aac242 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.571891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095338d-cb51-46ff-b958-8da207aac242 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.572155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7095338d-cb51-46ff-b958-8da207aac242 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.573426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 81/171] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:34.578863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095338d-cb51-46ff-b958-8da207aac242 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:34.600919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:34.706436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7095338d-cb51-46ff-b958-8da207aac242 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:34.750526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e2f22-3b82-458d-9435-66b8f895f7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:34.751511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e2f22-3b82-458d-9435-66b8f895f7a2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.751821 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e2f22-3b82-458d-9435-66b8f895f7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.752487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e2f22-3b82-458d-9435-66b8f895f7a2 None admin] PATH_INFO: `/v3/users/1a50181ffd4545db88d27e0331cd3f67` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.753956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e2f22-3b82-458d-9435-66b8f895f7a2 None admin] RBAC: Authorizing `identity:get_user(user_id=1a50181ffd4545db88d27e0331cd3f67)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:34.756783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e2f22-3b82-458d-9435-66b8f895f7a2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:34.758915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 82/172] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:34 2022] GET /identity/v3/users/1a50181ffd4545db88d27e0331cd3f67 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:34.759530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e2f22-3b82-458d-9435-66b8f895f7a2 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:34.795969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:34.804151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd23c1d-b3c1-4aef-91f5-447dbe4c3d0a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:34.850156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11c3aa-8e25-4df6-a44b-927bdc674f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:34.852163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11c3aa-8e25-4df6-a44b-927bdc674f72 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.852614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11c3aa-8e25-4df6-a44b-927bdc674f72 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.853044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11c3aa-8e25-4df6-a44b-927bdc674f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.883964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e2f22-3b82-458d-9435-66b8f895f7a2 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:34.924369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebe925-e0d8-4a4c-be82-bc400db37c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:34.925252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebe925-e0d8-4a4c-be82-bc400db37c5e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:34.925907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebe925-e0d8-4a4c-be82-bc400db37c5e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:34.926184 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebe925-e0d8-4a4c-be82-bc400db37c5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:34.935697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11c3aa-8e25-4df6-a44b-927bdc674f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:34.937228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11c3aa-8e25-4df6-a44b-927bdc674f72 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:34.940898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:35.004743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebe925-e0d8-4a4c-be82-bc400db37c5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:35.006635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebe925-e0d8-4a4c-be82-bc400db37c5e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:35.030914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:35.048103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 91/173] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:35.049374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea8c61-9da5-4ed5-b2fa-4ab59f78fc79 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.049654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea8c61-9da5-4ed5-b2fa-4ab59f78fc79 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.049923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea8c61-9da5-4ed5-b2fa-4ab59f78fc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.067303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-90ea8c61-9da5-4ed5-b2fa-4ab59f78fc79 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:35.068685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-90ea8c61-9da5-4ed5-b2fa-4ab59f78fc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:35.120694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 83/174] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:35.122112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda1625-4260-4c83-bbe0-45cd9bbc5c38 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.122465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda1625-4260-4c83-bbe0-45cd9bbc5c38 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.122810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda1625-4260-4c83-bbe0-45cd9bbc5c38 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.124308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 84/175] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:35.124922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda1625-4260-4c83-bbe0-45cd9bbc5c38 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:35.149191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 92/176] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:35.151009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299aa11-d0bd-4c44-9149-74b8ff59e437 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.151299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299aa11-d0bd-4c44-9149-74b8ff59e437 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.151573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299aa11-d0bd-4c44-9149-74b8ff59e437 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.153001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 93/177] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:35.153562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299aa11-d0bd-4c44-9149-74b8ff59e437 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:35.201077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:35.278164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda1625-4260-4c83-bbe0-45cd9bbc5c38 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:35.300873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299aa11-d0bd-4c44-9149-74b8ff59e437 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:35.303268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:35.314870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c4f50-88a8-4356-a1f0-ee2790740219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:35.315228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c4f50-88a8-4356-a1f0-ee2790740219 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.315524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c4f50-88a8-4356-a1f0-ee2790740219 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.315833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c4f50-88a8-4356-a1f0-ee2790740219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.344721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec819d72-c913-47b4-9a67-57fe9cfc4809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:35.345827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec819d72-c913-47b4-9a67-57fe9cfc4809 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.346388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec819d72-c913-47b4-9a67-57fe9cfc4809 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.346872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec819d72-c913-47b4-9a67-57fe9cfc4809 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.348426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec819d72-c913-47b4-9a67-57fe9cfc4809 None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:35.351712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec819d72-c913-47b4-9a67-57fe9cfc4809 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:35.353863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 94/178] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:35 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:35.355234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fdafe-6e29-40dc-bfe2-e4bc601f5a31 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.355778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fdafe-6e29-40dc-bfe2-e4bc601f5a31 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.356321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fdafe-6e29-40dc-bfe2-e4bc601f5a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.377937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-7f7fdafe-6e29-40dc-bfe2-e4bc601f5a31 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:35.379536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7f7fdafe-6e29-40dc-bfe2-e4bc601f5a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:35.389246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c4f50-88a8-4356-a1f0-ee2790740219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:35.390506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c4f50-88a8-4356-a1f0-ee2790740219 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:35.456267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 95/179] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:35.456967 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fdafe-6e29-40dc-bfe2-e4bc601f5a31 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:35.498046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:35.516404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 85/180] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:35.517596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666c593-e7f6-4fdf-8e07-b60117e3cb65 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.518046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666c593-e7f6-4fdf-8e07-b60117e3cb65 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.518335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666c593-e7f6-4fdf-8e07-b60117e3cb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.538505 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1666c593-e7f6-4fdf-8e07-b60117e3cb65 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:35.539761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1666c593-e7f6-4fdf-8e07-b60117e3cb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:35.558802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:35.571579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fdafe-6e29-40dc-bfe2-e4bc601f5a31 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:35.612829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614574c-ecf3-49c8-b965-90cc29ada87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:35.613385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614574c-ecf3-49c8-b965-90cc29ada87f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.613665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614574c-ecf3-49c8-b965-90cc29ada87f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.613936 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614574c-ecf3-49c8-b965-90cc29ada87f None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.615810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614574c-ecf3-49c8-b965-90cc29ada87f None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:35.617401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614574c-ecf3-49c8-b965-90cc29ada87f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:35.623750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 96/181] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:35 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:35.623750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614574c-ecf3-49c8-b965-90cc29ada87f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:35.626071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 86/182] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:35.627016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666c593-e7f6-4fdf-8e07-b60117e3cb65 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:35.790076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:35.793500 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666c593-e7f6-4fdf-8e07-b60117e3cb65 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:35.794402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614574c-ecf3-49c8-b965-90cc29ada87f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:35.833451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676abdf-47cd-454f-8598-3880320cd8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:35.835055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676abdf-47cd-454f-8598-3880320cd8f8 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.835347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676abdf-47cd-454f-8598-3880320cd8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.835622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676abdf-47cd-454f-8598-3880320cd8f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.845410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48354e16-2490-4696-93d4-d0cad0f2f14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:35.846529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48354e16-2490-4696-93d4-d0cad0f2f14d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:35.846651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676abdf-47cd-454f-8598-3880320cd8f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:35.847055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48354e16-2490-4696-93d4-d0cad0f2f14d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:35.847473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48354e16-2490-4696-93d4-d0cad0f2f14d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:35.848353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48354e16-2490-4696-93d4-d0cad0f2f14d None admin] RBAC: Adding query filter params (role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=af2562b4c9384f15b1d6a92b56581f77, user.id=1a50181ffd4545db88d27e0331cd3f67) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:35.848833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48354e16-2490-4696-93d4-d0cad0f2f14d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:35.851397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48354e16-2490-4696-93d4-d0cad0f2f14d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-a676abdf-47cd-454f-8598-3880320cd8f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:35.852910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:35.853952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:35.854740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 87/183] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:16:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:35.855385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676abdf-47cd-454f-8598-3880320cd8f8 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:35.861265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 97/184] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:35 2022] GET /identity/v3/role_assignments?user.id=1a50181ffd4545db88d27e0331cd3f67&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:35.862305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48354e16-2490-4696-93d4-d0cad0f2f14d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:35.875879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:35.979702 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676abdf-47cd-454f-8598-3880320cd8f8 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.016159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48354e16-2490-4696-93d4-d0cad0f2f14d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.018374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6475cf8-4920-4815-b72a-88f021468069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.019187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6475cf8-4920-4815-b72a-88f021468069 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.019634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6475cf8-4920-4815-b72a-88f021468069 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.020064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6475cf8-4920-4815-b72a-88f021468069 None admin] PATH_INFO: `/v3/projects/d08c7d83dc8447d0bfeb9920ad510b61` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.022008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6475cf8-4920-4815-b72a-88f021468069 None admin] RBAC: Authorizing `identity:get_project(project_id=d08c7d83dc8447d0bfeb9920ad510b61)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:36.023818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6475cf8-4920-4815-b72a-88f021468069 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:36.025743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 88/185] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:35 2022] GET /identity/v3/projects/d08c7d83dc8447d0bfeb9920ad510b61 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:36.026703 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6475cf8-4920-4815-b72a-88f021468069 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:36.058100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47d544-b645-420e-a407-b7669e2ffcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.058827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47d544-b645-420e-a407-b7669e2ffcfd None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.059113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47d544-b645-420e-a407-b7669e2ffcfd None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.059392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47d544-b645-420e-a407-b7669e2ffcfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.060156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47d544-b645-420e-a407-b7669e2ffcfd None admin] RBAC: Adding query filter params (role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=af2562b4c9384f15b1d6a92b56581f77, user.id=1a50181ffd4545db88d27e0331cd3f67) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:36.060449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47d544-b645-420e-a407-b7669e2ffcfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:36.062083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47d544-b645-420e-a407-b7669e2ffcfd None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:36.070100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 98/186] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:35 2022] GET /identity/v3/role_assignments?user.id=1a50181ffd4545db88d27e0331cd3f67&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:36.071127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47d544-b645-420e-a407-b7669e2ffcfd None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:36.129811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:36.136889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:36.155589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6475cf8-4920-4815-b72a-88f021468069 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.205014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95188b9-3b51-47bb-b6a7-e8fb3a29cce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.205676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95188b9-3b51-47bb-b6a7-e8fb3a29cce2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.205885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47d544-b645-420e-a407-b7669e2ffcfd None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.206013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95188b9-3b51-47bb-b6a7-e8fb3a29cce2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.206217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95188b9-3b51-47bb-b6a7-e8fb3a29cce2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.207714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95188b9-3b51-47bb-b6a7-e8fb3a29cce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:36.211690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95188b9-3b51-47bb-b6a7-e8fb3a29cce2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:36.214672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 89/187] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:36.215214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95188b9-3b51-47bb-b6a7-e8fb3a29cce2 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:36.271148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0cf9b-f652-4bee-a61e-1d72e8d35fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.272245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0cf9b-f652-4bee-a61e-1d72e8d35fac None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.272762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0cf9b-f652-4bee-a61e-1d72e8d35fac None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.273203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0cf9b-f652-4bee-a61e-1d72e8d35fac None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.274029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0cf9b-f652-4bee-a61e-1d72e8d35fac None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:36.274483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0cf9b-f652-4bee-a61e-1d72e8d35fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:36.275896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0cf9b-f652-4bee-a61e-1d72e8d35fac None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:36.285155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 99/188] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:36.285984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0cf9b-f652-4bee-a61e-1d72e8d35fac None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:36.345792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95188b9-3b51-47bb-b6a7-e8fb3a29cce2 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.382104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200cef83-d550-425f-b44f-46acce88f11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.382651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cef83-d550-425f-b44f-46acce88f11e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.382928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cef83-d550-425f-b44f-46acce88f11e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.383197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200cef83-d550-425f-b44f-46acce88f11e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.386291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:36.393565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200cef83-d550-425f-b44f-46acce88f11e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:36.397084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0cf9b-f652-4bee-a61e-1d72e8d35fac None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-200cef83-d550-425f-b44f-46acce88f11e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:36.399285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:36.400259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:36.401092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 90/189] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:16:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:36.401768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ce0f-df07-4e00-b92e-728c87aa3718 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.401833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ce0f-df07-4e00-b92e-728c87aa3718 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.402594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ce0f-df07-4e00-b92e-728c87aa3718 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.403573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 91/190] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:36.404258 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ce0f-df07-4e00-b92e-728c87aa3718 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:36.418287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:36.438929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee31a1b-64d6-45b1-ad6c-719aa66758ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.439536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee31a1b-64d6-45b1-ad6c-719aa66758ee None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.439813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee31a1b-64d6-45b1-ad6c-719aa66758ee None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.440084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee31a1b-64d6-45b1-ad6c-719aa66758ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.440842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee31a1b-64d6-45b1-ad6c-719aa66758ee None admin] RBAC: Adding query filter params (role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=d08c7d83dc8447d0bfeb9920ad510b61, user.id=1a50181ffd4545db88d27e0331cd3f67) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:36.441151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee31a1b-64d6-45b1-ad6c-719aa66758ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:36.442598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee31a1b-64d6-45b1-ad6c-719aa66758ee None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:36.449017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 100/191] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:36 2022] GET /identity/v3/role_assignments?user.id=1a50181ffd4545db88d27e0331cd3f67&scope.project.id=d08c7d83dc8447d0bfeb9920ad510b61&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:36.449799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee31a1b-64d6-45b1-ad6c-719aa66758ee None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:36.533406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ce0f-df07-4e00-b92e-728c87aa3718 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.571137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee31a1b-64d6-45b1-ad6c-719aa66758ee None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.577509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56da1c-5713-4e78-b499-8dabfb2ea462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.578137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56da1c-5713-4e78-b499-8dabfb2ea462 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.578416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56da1c-5713-4e78-b499-8dabfb2ea462 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.579190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56da1c-5713-4e78-b499-8dabfb2ea462 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.579382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56da1c-5713-4e78-b499-8dabfb2ea462 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:36.581119 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56da1c-5713-4e78-b499-8dabfb2ea462 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:36.606333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 92/192] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:16:36 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:36.608586 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56da1c-5713-4e78-b499-8dabfb2ea462 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:36.614338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fe463-01e4-47fe-a86d-62b4ac12f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.614998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fe463-01e4-47fe-a86d-62b4ac12f113 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.615899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fe463-01e4-47fe-a86d-62b4ac12f113 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.616243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fe463-01e4-47fe-a86d-62b4ac12f113 None admin] PATH_INFO: `/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.635954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fe463-01e4-47fe-a86d-62b4ac12f113 None admin] RBAC: Authorizing `identity:get_user(user_id=e654a6c3fda942e2a36c8f8cbd3d0aa6)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:36.639079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fe463-01e4-47fe-a86d-62b4ac12f113 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:36.641451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 101/193] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:36 2022] GET /identity/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:36.647759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f695b-8f8b-4583-b606-5194e097bebe None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.647759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f695b-8f8b-4583-b606-5194e097bebe None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.647759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f695b-8f8b-4583-b606-5194e097bebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.671740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d80f695b-8f8b-4583-b606-5194e097bebe None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:36.673037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d80f695b-8f8b-4583-b606-5194e097bebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:36.683440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:36.729067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:36.764255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 102/194] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:36.765253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f695b-8f8b-4583-b606-5194e097bebe None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:36.776477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56da1c-5713-4e78-b499-8dabfb2ea462 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.820668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad8f52-0c25-4e61-a369-00afc6a4f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.821291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad8f52-0c25-4e61-a369-00afc6a4f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.821572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad8f52-0c25-4e61-a369-00afc6a4f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.822007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad8f52-0c25-4e61-a369-00afc6a4f9d6 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.823051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad8f52-0c25-4e61-a369-00afc6a4f9d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:36.823363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad8f52-0c25-4e61-a369-00afc6a4f9d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:36.826605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad8f52-0c25-4e61-a369-00afc6a4f9d6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:36.838394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 93/195] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:16:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:36.841792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad8f52-0c25-4e61-a369-00afc6a4f9d6 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:36.889635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f695b-8f8b-4583-b606-5194e097bebe None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:36.931375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04558b31-61f6-4f60-a477-d940a62c74c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:36.932361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04558b31-61f6-4f60-a477-d940a62c74c6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:36.932866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04558b31-61f6-4f60-a477-d940a62c74c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:36.933197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04558b31-61f6-4f60-a477-d940a62c74c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:36.948524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:36.964455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad8f52-0c25-4e61-a369-00afc6a4f9d6 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:37.003173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:37.011895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfaa76-13e2-4712-ac30-4ec1d611d9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:37.012883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfaa76-13e2-4712-ac30-4ec1d611d9b2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:37.013166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfaa76-13e2-4712-ac30-4ec1d611d9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:37.013437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfaa76-13e2-4712-ac30-4ec1d611d9b2 None admin] PATH_INFO: `/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:37.014572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfaa76-13e2-4712-ac30-4ec1d611d9b2 None admin] RBAC: Authorizing `identity:get_user(user_id=e654a6c3fda942e2a36c8f8cbd3d0aa6)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:37.016032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfaa76-13e2-4712-ac30-4ec1d611d9b2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:37.017667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 94/196] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:36 2022] GET /identity/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:37.025642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfaa76-13e2-4712-ac30-4ec1d611d9b2 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:37.033879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04558b31-61f6-4f60-a477-d940a62c74c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:37.035551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04558b31-61f6-4f60-a477-d940a62c74c6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:37.150797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 103/197] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:37.151639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04558b31-61f6-4f60-a477-d940a62c74c6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:37.161515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfaa76-13e2-4712-ac30-4ec1d611d9b2 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:37.207049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:37.215059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8d561-af6d-4aa3-9677-39e0bd8a5156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:37.216247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8d561-af6d-4aa3-9677-39e0bd8a5156 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:37.216694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8d561-af6d-4aa3-9677-39e0bd8a5156 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:37.217127 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8d561-af6d-4aa3-9677-39e0bd8a5156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:37.274731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04558b31-61f6-4f60-a477-d940a62c74c6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:37.304509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:37.311999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244cf36-5505-49ed-9e16-b632010642ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:37.312672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244cf36-5505-49ed-9e16-b632010642ea None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:37.312953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244cf36-5505-49ed-9e16-b632010642ea None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:37.313292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244cf36-5505-49ed-9e16-b632010642ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:37.314620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8d561-af6d-4aa3-9677-39e0bd8a5156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:37.316727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8d561-af6d-4aa3-9677-39e0bd8a5156 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:37.328019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244cf36-5505-49ed-9e16-b632010642ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-7244cf36-5505-49ed-9e16-b632010642ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:37.340533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:37.341430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:37.342198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:37.343195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 104/198] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:37.344125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244cf36-5505-49ed-9e16-b632010642ea None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:37.436023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 95/199] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 00:16:37.436754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8d561-af6d-4aa3-9677-39e0bd8a5156 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:37.469226 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244cf36-5505-49ed-9e16-b632010642ea None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:37.471658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:37.508197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e42ad-55ce-4859-97c5-d7dae161e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:37.509111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e42ad-55ce-4859-97c5-d7dae161e8af None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:37.509567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e42ad-55ce-4859-97c5-d7dae161e8af None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:37.509982 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e42ad-55ce-4859-97c5-d7dae161e8af None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:37.511251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e42ad-55ce-4859-97c5-d7dae161e8af None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:37.512948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e42ad-55ce-4859-97c5-d7dae161e8af None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:37.516023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 105/200] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:37 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:37.517228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e42ad-55ce-4859-97c5-d7dae161e8af None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:37.554389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:37.561747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8d561-af6d-4aa3-9677-39e0bd8a5156 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:37.602460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5886bcb-497d-404d-b02f-5dd067f24e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:37.603457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5886bcb-497d-404d-b02f-5dd067f24e43 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:37.603903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5886bcb-497d-404d-b02f-5dd067f24e43 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:37.604316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5886bcb-497d-404d-b02f-5dd067f24e43 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:37.605154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5886bcb-497d-404d-b02f-5dd067f24e43 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:37.605668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5886bcb-497d-404d-b02f-5dd067f24e43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:37.607208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5886bcb-497d-404d-b02f-5dd067f24e43 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:37.617151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 96/201] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:37.618274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5886bcb-497d-404d-b02f-5dd067f24e43 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:37.639120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e42ad-55ce-4859-97c5-d7dae161e8af None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:37.684250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a9ffc-7f57-4fa6-af9c-e99c938da4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:37.685324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a9ffc-7f57-4fa6-af9c-e99c938da4d2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:37.685593 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a9ffc-7f57-4fa6-af9c-e99c938da4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:37.685870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a9ffc-7f57-4fa6-af9c-e99c938da4d2 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:37.687212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a9ffc-7f57-4fa6-af9c-e99c938da4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:37.689267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a9ffc-7f57-4fa6-af9c-e99c938da4d2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:37.691146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 106/202] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:37 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:37.691899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a9ffc-7f57-4fa6-af9c-e99c938da4d2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:37.732359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5886bcb-497d-404d-b02f-5dd067f24e43 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:37.743590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:37.772421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aabfe-d5d2-49b3-8a32-6a7ef51d68bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:37.773405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aabfe-d5d2-49b3-8a32-6a7ef51d68bb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:37.773840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aabfe-d5d2-49b3-8a32-6a7ef51d68bb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:37.774253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aabfe-d5d2-49b3-8a32-6a7ef51d68bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:37.775162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6aabfe-d5d2-49b3-8a32-6a7ef51d68bb None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=9a8fffb476e54c20b9662b528c1868ba, user.id=e654a6c3fda942e2a36c8f8cbd3d0aa6) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:37.775655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6aabfe-d5d2-49b3-8a32-6a7ef51d68bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:37.777538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6aabfe-d5d2-49b3-8a32-6a7ef51d68bb None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:37.784194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 97/203] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:37 2022] GET /identity/v3/role_assignments?user.id=e654a6c3fda942e2a36c8f8cbd3d0aa6&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:37.785233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aabfe-d5d2-49b3-8a32-6a7ef51d68bb None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:37.820916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a9ffc-7f57-4fa6-af9c-e99c938da4d2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:37.821521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:37.884856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dfc26-a227-467c-853d-2e08611ec568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:37.885787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dfc26-a227-467c-853d-2e08611ec568 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:37.886228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dfc26-a227-467c-853d-2e08611ec568 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:37.886643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dfc26-a227-467c-853d-2e08611ec568 None admin] PATH_INFO: `/v3/users/ee799e01b80643ab827342efcfc1c01e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:37.888295 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dfc26-a227-467c-853d-2e08611ec568 None admin] RBAC: Authorizing `identity:get_user(user_id=ee799e01b80643ab827342efcfc1c01e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:37.890660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dfc26-a227-467c-853d-2e08611ec568 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:37.893125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 107/204] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:37 2022] GET /identity/v3/users/ee799e01b80643ab827342efcfc1c01e => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:37.901099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dfc26-a227-467c-853d-2e08611ec568 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:38.018270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aabfe-d5d2-49b3-8a32-6a7ef51d68bb None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:38.034137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dfc26-a227-467c-853d-2e08611ec568 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:38.044714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:38.067414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a335a6-e5b1-415c-8c73-e7e93d562149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:38.068006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a335a6-e5b1-415c-8c73-e7e93d562149 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:38.068362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a335a6-e5b1-415c-8c73-e7e93d562149 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:38.068808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a335a6-e5b1-415c-8c73-e7e93d562149 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:38.069783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a335a6-e5b1-415c-8c73-e7e93d562149 None admin] RBAC: Adding query filter params (role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=9a8fffb476e54c20b9662b528c1868ba, user.id=e654a6c3fda942e2a36c8f8cbd3d0aa6) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:38.070097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a335a6-e5b1-415c-8c73-e7e93d562149 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:38.071774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a335a6-e5b1-415c-8c73-e7e93d562149 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:38.079738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389b50e-ea50-4b18-9ce0-aeda9bf6db26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:38.080814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389b50e-ea50-4b18-9ce0-aeda9bf6db26 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:38.081298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389b50e-ea50-4b18-9ce0-aeda9bf6db26 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:38.081769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389b50e-ea50-4b18-9ce0-aeda9bf6db26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:38.089557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 98/205] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:37 2022] GET /identity/v3/role_assignments?user.id=e654a6c3fda942e2a36c8f8cbd3d0aa6&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:38.175983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389b50e-ea50-4b18-9ce0-aeda9bf6db26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:38.178522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389b50e-ea50-4b18-9ce0-aeda9bf6db26 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:38.316174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 108/206] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:38.321782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a335a6-e5b1-415c-8c73-e7e93d562149 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:38.433748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:38.451806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a335a6-e5b1-415c-8c73-e7e93d562149 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:38.494432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc2e77-be32-4fb7-bfaa-26ed3bf560ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:38.495075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc2e77-be32-4fb7-bfaa-26ed3bf560ee None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:38.495352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc2e77-be32-4fb7-bfaa-26ed3bf560ee None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:38.495713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc2e77-be32-4fb7-bfaa-26ed3bf560ee None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:38.501254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc2e77-be32-4fb7-bfaa-26ed3bf560ee None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:38.502997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc2e77-be32-4fb7-bfaa-26ed3bf560ee None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:38.504666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 99/207] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:38 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:38.511181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389b50e-ea50-4b18-9ce0-aeda9bf6db26 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:38.519589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:38.632566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389b50e-ea50-4b18-9ce0-aeda9bf6db26 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:38.668823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784c2e8-5633-45a4-932c-4a0c0488b389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:38.669416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784c2e8-5633-45a4-932c-4a0c0488b389 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:38.669688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784c2e8-5633-45a4-932c-4a0c0488b389 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:38.669957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784c2e8-5633-45a4-932c-4a0c0488b389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:38.670680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784c2e8-5633-45a4-932c-4a0c0488b389 None admin] RBAC: Adding query filter params (role.id=a6588e7acf534502a191a0bca0d46c39, scope.project.id=af2562b4c9384f15b1d6a92b56581f77, user.id=ee799e01b80643ab827342efcfc1c01e) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:38.670984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784c2e8-5633-45a4-932c-4a0c0488b389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:38.672434 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784c2e8-5633-45a4-932c-4a0c0488b389 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:38.678988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 109/208] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:38 2022] GET /identity/v3/role_assignments?user.id=ee799e01b80643ab827342efcfc1c01e&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=a6588e7acf534502a191a0bca0d46c39 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:38.951120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccc163-bc4a-4ba0-aa18-10cb5bddd769 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:38.951463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccc163-bc4a-4ba0-aa18-10cb5bddd769 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:38.951574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccc163-bc4a-4ba0-aa18-10cb5bddd769 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:38.953240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 100/209] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:38.958511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d63c8e-2b40-4c17-96cc-3bed7319f490 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:38.959519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d63c8e-2b40-4c17-96cc-3bed7319f490 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:38.959595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d63c8e-2b40-4c17-96cc-3bed7319f490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:38.978688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-69d63c8e-2b40-4c17-96cc-3bed7319f490 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:38.979887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-69d63c8e-2b40-4c17-96cc-3bed7319f490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:39.039814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e62ff5-1865-4ccf-86bd-e8ef0d20207b None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:39.040453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e62ff5-1865-4ccf-86bd-e8ef0d20207b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:39.040453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e62ff5-1865-4ccf-86bd-e8ef0d20207b None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:39.041940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 101/210] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:39.046956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e548f2b-e19e-4789-ac0c-f40472b2dc8e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:39.047241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e548f2b-e19e-4789-ac0c-f40472b2dc8e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:39.047519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e548f2b-e19e-4789-ac0c-f40472b2dc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:39.050549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:39.056737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 110/211] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:39.073845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0e548f2b-e19e-4789-ac0c-f40472b2dc8e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:39.075311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0e548f2b-e19e-4789-ac0c-f40472b2dc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:39.152972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 102/212] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:39.360266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d63c8e-2b40-4c17-96cc-3bed7319f490 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:39.451646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e548f2b-e19e-4789-ac0c-f40472b2dc8e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:39.463945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d63c8e-2b40-4c17-96cc-3bed7319f490 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:39.502070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35857f9-bcee-4978-9d13-bd36ccf5f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:39.502875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35857f9-bcee-4978-9d13-bd36ccf5f586 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:39.503243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35857f9-bcee-4978-9d13-bd36ccf5f586 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:39.503611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35857f9-bcee-4978-9d13-bd36ccf5f586 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:39.515081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35857f9-bcee-4978-9d13-bd36ccf5f586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-c35857f9-bcee-4978-9d13-bd36ccf5f586 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:39.520145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:39.521030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:39.522445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 111/213] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:39.528418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35857f9-bcee-4978-9d13-bd36ccf5f586 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:39.561394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:39.581981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e548f2b-e19e-4789-ac0c-f40472b2dc8e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:39.607115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:39.627930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bba2e7-c0b3-4585-b926-710ee1e64c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:39.628945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bba2e7-c0b3-4585-b926-710ee1e64c0c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:39.629365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bba2e7-c0b3-4585-b926-710ee1e64c0c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:39.629826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bba2e7-c0b3-4585-b926-710ee1e64c0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:39.631142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bba2e7-c0b3-4585-b926-710ee1e64c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:39.633005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bba2e7-c0b3-4585-b926-710ee1e64c0c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:39.634784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 103/214] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:39.643828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bba2e7-c0b3-4585-b926-710ee1e64c0c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:39.652587 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35857f9-bcee-4978-9d13-bd36ccf5f586 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:39.689387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d022fd-d73b-48bf-b45c-9c362ec94641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:39.690141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d022fd-d73b-48bf-b45c-9c362ec94641 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:39.690419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d022fd-d73b-48bf-b45c-9c362ec94641 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:39.690694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d022fd-d73b-48bf-b45c-9c362ec94641 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:39.691491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d022fd-d73b-48bf-b45c-9c362ec94641 None admin] RBAC: Adding query filter params (name=member) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:39.691789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d022fd-d73b-48bf-b45c-9c362ec94641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:39.693258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d022fd-d73b-48bf-b45c-9c362ec94641 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:39.706868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 112/215] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:39.708067 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d022fd-d73b-48bf-b45c-9c362ec94641 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:39.748940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bba2e7-c0b3-4585-b926-710ee1e64c0c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:39.785592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a3bfc-fdfc-4894-83a7-50d895295f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:39.786135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a3bfc-fdfc-4894-83a7-50d895295f70 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:39.786413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a3bfc-fdfc-4894-83a7-50d895295f70 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:39.786685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a3bfc-fdfc-4894-83a7-50d895295f70 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:39.787403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a3bfc-fdfc-4894-83a7-50d895295f70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:39.789044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a3bfc-fdfc-4894-83a7-50d895295f70 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:39.817663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d022fd-d73b-48bf-b45c-9c362ec94641 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:39.820921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 104/216] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:16:39 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:39.844389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:39.858530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e716be3-02e0-4b7e-8650-e704e6cdbde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:39.859179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e716be3-02e0-4b7e-8650-e704e6cdbde7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:39.859456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e716be3-02e0-4b7e-8650-e704e6cdbde7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:39.859726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e716be3-02e0-4b7e-8650-e704e6cdbde7 None admin] PATH_INFO: `/v3/users/3d99e94bd2c34f3dbb9b004565d16b88` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:39.875742 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e716be3-02e0-4b7e-8650-e704e6cdbde7 None admin] RBAC: Authorizing `identity:get_user(user_id=3d99e94bd2c34f3dbb9b004565d16b88)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:39.877344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e716be3-02e0-4b7e-8650-e704e6cdbde7 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:39.884485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 113/217] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:39 2022] GET /identity/v3/users/3d99e94bd2c34f3dbb9b004565d16b88 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:39.889664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a3bfc-fdfc-4894-83a7-50d895295f70 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:39.903753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:40.008183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a3bfc-fdfc-4894-83a7-50d895295f70 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:40.044721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a5d04-a0dd-46a8-90d4-02e8fc39274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:40.045440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a5d04-a0dd-46a8-90d4-02e8fc39274e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:40.045675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a5d04-a0dd-46a8-90d4-02e8fc39274e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:40.045951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a5d04-a0dd-46a8-90d4-02e8fc39274e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:40.069902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398349-8bf0-45b6-aac4-4c1a4a0296a2 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:40.070733 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398349-8bf0-45b6-aac4-4c1a4a0296a2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:40.071001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398349-8bf0-45b6-aac4-4c1a4a0296a2 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:40.072594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 114/218] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:40.077919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08882733-6d84-4cba-926d-e1d1190ca89a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:40.078308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08882733-6d84-4cba-926d-e1d1190ca89a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:40.078780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08882733-6d84-4cba-926d-e1d1190ca89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:40.101266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-08882733-6d84-4cba-926d-e1d1190ca89a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:40.102979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-08882733-6d84-4cba-926d-e1d1190ca89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:40.129499 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a5d04-a0dd-46a8-90d4-02e8fc39274e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:40.131523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a5d04-a0dd-46a8-90d4-02e8fc39274e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:40.144957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:40.199887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 115/219] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:40.255370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 105/220] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:40.258491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08882733-6d84-4cba-926d-e1d1190ca89a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:40.367199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08882733-6d84-4cba-926d-e1d1190ca89a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:40.397135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:40.404961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebf8eb-5ed6-41da-9f38-bd4856419b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:40.406092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebf8eb-5ed6-41da-9f38-bd4856419b52 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:40.406614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebf8eb-5ed6-41da-9f38-bd4856419b52 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:40.407081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebf8eb-5ed6-41da-9f38-bd4856419b52 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:40.408427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebf8eb-5ed6-41da-9f38-bd4856419b52 None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:40.410119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebf8eb-5ed6-41da-9f38-bd4856419b52 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:40.416597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 116/221] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:40 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:40.418726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a5d04-a0dd-46a8-90d4-02e8fc39274e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:40.522805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a5d04-a0dd-46a8-90d4-02e8fc39274e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:40.533039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:40.559810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708efeef-91b4-45f6-84b9-672393921e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:40.560837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708efeef-91b4-45f6-84b9-672393921e9b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:40.561334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708efeef-91b4-45f6-84b9-672393921e9b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:40.561801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708efeef-91b4-45f6-84b9-672393921e9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:40.562738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708efeef-91b4-45f6-84b9-672393921e9b None admin] RBAC: Adding query filter params (role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=9a8fffb476e54c20b9662b528c1868ba, user.id=3d99e94bd2c34f3dbb9b004565d16b88) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:40.563249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708efeef-91b4-45f6-84b9-672393921e9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:40.564920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708efeef-91b4-45f6-84b9-672393921e9b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:40.576584 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 106/222] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:40 2022] GET /identity/v3/role_assignments?user.id=3d99e94bd2c34f3dbb9b004565d16b88&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:40.632162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebf8eb-5ed6-41da-9f38-bd4856419b52 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:40.692109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fef91-9a12-4818-84ea-fb1458286b30 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:40.692795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fef91-9a12-4818-84ea-fb1458286b30 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:40.693812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fef91-9a12-4818-84ea-fb1458286b30 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:40.696298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 107/223] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:40.701369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7192b8f-5656-4223-a6d7-ef1b9c8468ef None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:40.701933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7192b8f-5656-4223-a6d7-ef1b9c8468ef None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:40.702412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7192b8f-5656-4223-a6d7-ef1b9c8468ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:40.726327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b7192b8f-5656-4223-a6d7-ef1b9c8468ef None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:40.728449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b7192b8f-5656-4223-a6d7-ef1b9c8468ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:40.779425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebf8eb-5ed6-41da-9f38-bd4856419b52 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:40.819398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 108/224] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:40.827770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fb40a-2edd-44ef-875c-3df8860f5c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:40.828748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fb40a-2edd-44ef-875c-3df8860f5c73 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:40.829248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fb40a-2edd-44ef-875c-3df8860f5c73 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:40.829717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fb40a-2edd-44ef-875c-3df8860f5c73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:40.840803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fb40a-2edd-44ef-875c-3df8860f5c73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-3b8fb40a-2edd-44ef-875c-3df8860f5c73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:40.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:40.846848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:40.847622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:40.850708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 117/225] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:16:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:40.857198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7192b8f-5656-4223-a6d7-ef1b9c8468ef None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:40.982077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7192b8f-5656-4223-a6d7-ef1b9c8468ef None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:40.985249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:41.017961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df940c88-3fc2-49e9-8fa2-93ce28813e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:41.018923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df940c88-3fc2-49e9-8fa2-93ce28813e23 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:41.019400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df940c88-3fc2-49e9-8fa2-93ce28813e23 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:41.019878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df940c88-3fc2-49e9-8fa2-93ce28813e23 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:41.020773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df940c88-3fc2-49e9-8fa2-93ce28813e23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:41.021835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df940c88-3fc2-49e9-8fa2-93ce28813e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:41.023470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df940c88-3fc2-49e9-8fa2-93ce28813e23 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:41.031993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 109/226] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:41.037505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fb40a-2edd-44ef-875c-3df8860f5c73 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:41.098891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:41.104684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df940c88-3fc2-49e9-8fa2-93ce28813e23 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:41.167061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fb40a-2edd-44ef-875c-3df8860f5c73 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:41.212173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96958ba3-1052-474e-9f86-b503616936b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:41.212838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96958ba3-1052-474e-9f86-b503616936b2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:41.213111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96958ba3-1052-474e-9f86-b503616936b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:41.213381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96958ba3-1052-474e-9f86-b503616936b2 None admin] PATH_INFO: `/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:41.214535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96958ba3-1052-474e-9f86-b503616936b2 None admin] RBAC: Authorizing `identity:get_user(user_id=e654a6c3fda942e2a36c8f8cbd3d0aa6)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:41.215989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96958ba3-1052-474e-9f86-b503616936b2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:41.217591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 118/227] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:41 2022] GET /identity/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:41.231012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96958ba3-1052-474e-9f86-b503616936b2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:41.331784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df940c88-3fc2-49e9-8fa2-93ce28813e23 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:41.378546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363d6f0-5f51-48e7-9337-a4ef0f53133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:41.379219 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363d6f0-5f51-48e7-9337-a4ef0f53133d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:41.379496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363d6f0-5f51-48e7-9337-a4ef0f53133d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:41.379769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363d6f0-5f51-48e7-9337-a4ef0f53133d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:41.403236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363d6f0-5f51-48e7-9337-a4ef0f53133d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-d363d6f0-5f51-48e7-9337-a4ef0f53133d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:41.409635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:41.410501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:41.411280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 110/228] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:16:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:41.425306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96958ba3-1052-474e-9f86-b503616936b2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:41.425534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363d6f0-5f51-48e7-9337-a4ef0f53133d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:41.460833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9555-0efb-4247-bf30-0b1192f841e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:41.461508 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9555-0efb-4247-bf30-0b1192f841e7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:41.461784 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9555-0efb-4247-bf30-0b1192f841e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:41.462050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9555-0efb-4247-bf30-0b1192f841e7 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:41.463232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d9555-0efb-4247-bf30-0b1192f841e7 None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:41.464788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d9555-0efb-4247-bf30-0b1192f841e7 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:41.466568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 119/229] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:41 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:41.466906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:41.472473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9555-0efb-4247-bf30-0b1192f841e7 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:41.497659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:41.576969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9555-0efb-4247-bf30-0b1192f841e7 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:41.612168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a4bd6-cec0-4be8-ae7a-14b697e1b969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:41.612858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a4bd6-cec0-4be8-ae7a-14b697e1b969 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:41.613134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a4bd6-cec0-4be8-ae7a-14b697e1b969 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:41.613403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a4bd6-cec0-4be8-ae7a-14b697e1b969 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba/users/e654a6c3fda942e2a36c8f8cbd3d0aa6/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:41.615917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a4bd6-cec0-4be8-ae7a-14b697e1b969 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a8fffb476e54c20b9662b528c1868ba, user_id=e654a6c3fda942e2a36c8f8cbd3d0aa6, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:41.616850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a4bd6-cec0-4be8-ae7a-14b697e1b969 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:41.638251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 120/230] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:16:41 2022] PUT /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba/users/e654a6c3fda942e2a36c8f8cbd3d0aa6/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:16:41.699014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363d6f0-5f51-48e7-9337-a4ef0f53133d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:41.750813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9d763-18dc-4513-abf3-a6d54442e791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:41.751358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9d763-18dc-4513-abf3-a6d54442e791 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:41.751723 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9d763-18dc-4513-abf3-a6d54442e791 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:41.751974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9d763-18dc-4513-abf3-a6d54442e791 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:41.752795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b9d763-18dc-4513-abf3-a6d54442e791 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:41.753174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b9d763-18dc-4513-abf3-a6d54442e791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:41.754702 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b9d763-18dc-4513-abf3-a6d54442e791 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:41.769041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 111/231] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:16:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:41.775406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a4bd6-cec0-4be8-ae7a-14b697e1b969 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:41.817863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:41.890955 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a4bd6-cec0-4be8-ae7a-14b697e1b969 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:41.929632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953f21c-3028-4e25-960b-0a480860d4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:41.930174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953f21c-3028-4e25-960b-0a480860d4a6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:41.930448 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953f21c-3028-4e25-960b-0a480860d4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:41.930712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953f21c-3028-4e25-960b-0a480860d4a6 None admin] PATH_INFO: `/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:41.931838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953f21c-3028-4e25-960b-0a480860d4a6 None admin] RBAC: Authorizing `identity:get_user(user_id=e654a6c3fda942e2a36c8f8cbd3d0aa6)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:41.933330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953f21c-3028-4e25-960b-0a480860d4a6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:41.935326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 121/232] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:41 2022] GET /identity/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:41.940970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9d763-18dc-4513-abf3-a6d54442e791 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:42.035824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:42.055086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9d763-18dc-4513-abf3-a6d54442e791 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:42.090917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df61870-e3bd-4275-b59b-2a42dfcfe11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:42.091611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df61870-e3bd-4275-b59b-2a42dfcfe11b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:42.091887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df61870-e3bd-4275-b59b-2a42dfcfe11b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:42.092202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df61870-e3bd-4275-b59b-2a42dfcfe11b None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:42.093418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df61870-e3bd-4275-b59b-2a42dfcfe11b None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:42.094874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df61870-e3bd-4275-b59b-2a42dfcfe11b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:42.097216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 112/233] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:41 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:42.103343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953f21c-3028-4e25-960b-0a480860d4a6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:42.234044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953f21c-3028-4e25-960b-0a480860d4a6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:42.235673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:42.289814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdc88f-8a93-4338-917b-0014fdab764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:42.290535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdc88f-8a93-4338-917b-0014fdab764d None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:42.290819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdc88f-8a93-4338-917b-0014fdab764d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:42.291096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdc88f-8a93-4338-917b-0014fdab764d None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba/users/e654a6c3fda942e2a36c8f8cbd3d0aa6/roles/2986c98e353a489592c69ada5c3c2aaf` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:42.298362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdc88f-8a93-4338-917b-0014fdab764d None admin] RBAC: Authorizing `identity:create_grant(project_id=9a8fffb476e54c20b9662b528c1868ba, user_id=e654a6c3fda942e2a36c8f8cbd3d0aa6, role_id=2986c98e353a489592c69ada5c3c2aaf)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:42.299852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fdc88f-8a93-4338-917b-0014fdab764d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:42.308129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144c27-7e09-42d1-80ac-9f97e29308bc None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:42.308700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144c27-7e09-42d1-80ac-9f97e29308bc None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:42.308700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144c27-7e09-42d1-80ac-9f97e29308bc None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:42.311270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 113/234] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:42.315390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba6eed-05bc-4e53-ac93-64996a8f1c0e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:42.315677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba6eed-05bc-4e53-ac93-64996a8f1c0e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:42.315949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba6eed-05bc-4e53-ac93-64996a8f1c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:42.325801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 122/235] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:16:42 2022] PUT /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba/users/e654a6c3fda942e2a36c8f8cbd3d0aa6/roles/2986c98e353a489592c69ada5c3c2aaf => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:16:42.345328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d6305-85bd-4bf0-b1e2-23c88b43fd0b None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:42.345999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d6305-85bd-4bf0-b1e2-23c88b43fd0b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:42.346502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d6305-85bd-4bf0-b1e2-23c88b43fd0b None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:42.348520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 123/236] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:42.354014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b3ba6eed-05bc-4e53-ac93-64996a8f1c0e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:42.355282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b3ba6eed-05bc-4e53-ac93-64996a8f1c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:42.355775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504761e-013b-4755-a54a-6a88044d3774 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:42.356260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504761e-013b-4755-a54a-6a88044d3774 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:42.356866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504761e-013b-4755-a54a-6a88044d3774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:42.383081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b504761e-013b-4755-a54a-6a88044d3774 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:42.384287 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b504761e-013b-4755-a54a-6a88044d3774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:42.498766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 114/237] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:42.513112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 124/238] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:42.782178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba6eed-05bc-4e53-ac93-64996a8f1c0e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:42.847864 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504761e-013b-4755-a54a-6a88044d3774 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:42.893150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:42.932400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba6eed-05bc-4e53-ac93-64996a8f1c0e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:42.959553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:42.973456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504761e-013b-4755-a54a-6a88044d3774 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:43.018887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85252717-07da-4737-bac2-9b8fc265e214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:43.019530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85252717-07da-4737-bac2-9b8fc265e214 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.019806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85252717-07da-4737-bac2-9b8fc265e214 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.020076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85252717-07da-4737-bac2-9b8fc265e214 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.021748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85252717-07da-4737-bac2-9b8fc265e214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:43.023214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85252717-07da-4737-bac2-9b8fc265e214 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:43.023705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7532a-1018-4e79-9a91-dc1694155d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:43.024554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7532a-1018-4e79-9a91-dc1694155d6f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.025020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7532a-1018-4e79-9a91-dc1694155d6f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.025166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 125/239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:43.025595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7532a-1018-4e79-9a91-dc1694155d6f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.026374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec171e8-50f6-421a-937f-5ff9bd0dcb53 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.026643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec171e8-50f6-421a-937f-5ff9bd0dcb53 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.026913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec171e8-50f6-421a-937f-5ff9bd0dcb53 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.028260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 126/240] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:43.032846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec171e8-50f6-421a-937f-5ff9bd0dcb53 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:43.041110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7532a-1018-4e79-9a91-dc1694155d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-40c7532a-1018-4e79-9a91-dc1694155d6f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:43.046556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:43.047425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:43.048198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 115/241] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:43.054565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d2fa0-82d2-4193-976a-1d8ff8383b60 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.054849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d2fa0-82d2-4193-976a-1d8ff8383b60 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.055123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d2fa0-82d2-4193-976a-1d8ff8383b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.074890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-761d2fa0-82d2-4193-976a-1d8ff8383b60 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:43.076067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-761d2fa0-82d2-4193-976a-1d8ff8383b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:43.145467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec171e8-50f6-421a-937f-5ff9bd0dcb53 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:43.156711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 116/242] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:43.157611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d2fa0-82d2-4193-976a-1d8ff8383b60 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:43.181542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9c42b-611e-4621-8b2a-6034106f4936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:43.182143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9c42b-611e-4621-8b2a-6034106f4936 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.182417 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9c42b-611e-4621-8b2a-6034106f4936 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.182687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9c42b-611e-4621-8b2a-6034106f4936 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.183326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9c42b-611e-4621-8b2a-6034106f4936 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:43.184795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9c42b-611e-4621-8b2a-6034106f4936 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:43.217408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 127/243] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:16:43 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:43.304951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:43.341160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d2fa0-82d2-4193-976a-1d8ff8383b60 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:43.387535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85a592-fbbd-4468-91a4-92c7566a9554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:43.388166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85a592-fbbd-4468-91a4-92c7566a9554 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.388445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85a592-fbbd-4468-91a4-92c7566a9554 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.388753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85a592-fbbd-4468-91a4-92c7566a9554 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.389441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85a592-fbbd-4468-91a4-92c7566a9554 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:43.389741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85a592-fbbd-4468-91a4-92c7566a9554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:43.391188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85a592-fbbd-4468-91a4-92c7566a9554 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:43.401403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 117/244] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:43.406916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9c42b-611e-4621-8b2a-6034106f4936 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:43.429425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:43.526595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9c42b-611e-4621-8b2a-6034106f4936 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:43.571530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62231-cdca-4802-a3a3-f0821768a791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:43.572093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62231-cdca-4802-a3a3-f0821768a791 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.572371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62231-cdca-4802-a3a3-f0821768a791 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.572670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62231-cdca-4802-a3a3-f0821768a791 None admin] PATH_INFO: `/v3/users/6b23de1a521e481dab26f9905b088761` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.577276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85a592-fbbd-4468-91a4-92c7566a9554 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:43.590612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e62231-cdca-4802-a3a3-f0821768a791 None admin] RBAC: Authorizing `identity:get_user(user_id=6b23de1a521e481dab26f9905b088761)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:43.592800 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e62231-cdca-4802-a3a3-f0821768a791 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:43.595295 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 128/245] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:43 2022] GET /identity/v3/users/6b23de1a521e481dab26f9905b088761 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:43.602321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62231-cdca-4802-a3a3-f0821768a791 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:43.709525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:43.728181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85a592-fbbd-4468-91a4-92c7566a9554 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:43.742023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62231-cdca-4802-a3a3-f0821768a791 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:43.772797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c2fd4-0118-45a0-9919-6408f4f727ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:43.773444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c2fd4-0118-45a0-9919-6408f4f727ac None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.773722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c2fd4-0118-45a0-9919-6408f4f727ac None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c2fd4-0118-45a0-9919-6408f4f727ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.776936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:43.784288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf887ba0-f0e7-4230-8078-c2d7e5f67ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:43.785042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf887ba0-f0e7-4230-8078-c2d7e5f67ad5 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.785322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf887ba0-f0e7-4230-8078-c2d7e5f67ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.785592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf887ba0-f0e7-4230-8078-c2d7e5f67ad5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.791297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c2fd4-0118-45a0-9919-6408f4f727ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-929c2fd4-0118-45a0-9919-6408f4f727ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:43.797011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:43.797962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:43.798752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:43.801553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 118/246] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:43.807889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c2fd4-0118-45a0-9919-6408f4f727ac None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:43.868107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf887ba0-f0e7-4230-8078-c2d7e5f67ad5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:43.869563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf887ba0-f0e7-4230-8078-c2d7e5f67ad5 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:43.945756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c2fd4-0118-45a0-9919-6408f4f727ac None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:43.952762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:43.979586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 129/247] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 00:16:43.984604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7fcbb-5daa-4864-a514-00e92cc8cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:43.985027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf887ba0-f0e7-4230-8078-c2d7e5f67ad5 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:43.985581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7fcbb-5daa-4864-a514-00e92cc8cc5f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:43.986041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7fcbb-5daa-4864-a514-00e92cc8cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:43.986450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7fcbb-5daa-4864-a514-00e92cc8cc5f None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:43.987255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7fcbb-5daa-4864-a514-00e92cc8cc5f None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:43.987710 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7fcbb-5daa-4864-a514-00e92cc8cc5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:43.989408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7fcbb-5daa-4864-a514-00e92cc8cc5f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:43.998955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 119/248] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:44.004657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7fcbb-5daa-4864-a514-00e92cc8cc5f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:44.069093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:44.093616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf887ba0-f0e7-4230-8078-c2d7e5f67ad5 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:44.122844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7fcbb-5daa-4864-a514-00e92cc8cc5f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:44.135448 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dda202-3ca3-45d2-97d8-56c383983aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:44.135998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dda202-3ca3-45d2-97d8-56c383983aac None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:44.136346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dda202-3ca3-45d2-97d8-56c383983aac None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:44.136731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dda202-3ca3-45d2-97d8-56c383983aac None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:44.138039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dda202-3ca3-45d2-97d8-56c383983aac None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:44.139555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dda202-3ca3-45d2-97d8-56c383983aac None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:44.141412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 130/249] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:43 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:44.147954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dda202-3ca3-45d2-97d8-56c383983aac None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:44.167415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db13d4b-17e9-41b5-924b-03e0b99cc816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:44.168832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13d4b-17e9-41b5-924b-03e0b99cc816 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:44.169256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13d4b-17e9-41b5-924b-03e0b99cc816 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:44.169672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13d4b-17e9-41b5-924b-03e0b99cc816 None admin] PATH_INFO: `/v3/users/3d99e94bd2c34f3dbb9b004565d16b88` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:44.170973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db13d4b-17e9-41b5-924b-03e0b99cc816 None admin] RBAC: Authorizing `identity:get_user(user_id=3d99e94bd2c34f3dbb9b004565d16b88)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:44.173442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db13d4b-17e9-41b5-924b-03e0b99cc816 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:44.175961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 120/250] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:44 2022] GET /identity/v3/users/3d99e94bd2c34f3dbb9b004565d16b88 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:44.182828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db13d4b-17e9-41b5-924b-03e0b99cc816 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:44.208987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:44.260734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dda202-3ca3-45d2-97d8-56c383983aac None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:44.297290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70a67d-7feb-443b-93c0-8e2fec2d50fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:44.298055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70a67d-7feb-443b-93c0-8e2fec2d50fc None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:44.298424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70a67d-7feb-443b-93c0-8e2fec2d50fc None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:44.298862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70a67d-7feb-443b-93c0-8e2fec2d50fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:44.300658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70a67d-7feb-443b-93c0-8e2fec2d50fc None admin] RBAC: Adding query filter params (role.id=a6588e7acf534502a191a0bca0d46c39, scope.project.id=9a8fffb476e54c20b9662b528c1868ba, user.id=6b23de1a521e481dab26f9905b088761) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:44.300658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70a67d-7feb-443b-93c0-8e2fec2d50fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:44.302001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70a67d-7feb-443b-93c0-8e2fec2d50fc None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:44.313076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 131/251] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:44 2022] GET /identity/v3/role_assignments?user.id=6b23de1a521e481dab26f9905b088761&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=a6588e7acf534502a191a0bca0d46c39 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:44.355142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db13d4b-17e9-41b5-924b-03e0b99cc816 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:44.393557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:44.400322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c67b8-8a4d-4b30-85a2-3326ae8094c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:44.401076 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c67b8-8a4d-4b30-85a2-3326ae8094c9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:44.401374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c67b8-8a4d-4b30-85a2-3326ae8094c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:44.401727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c67b8-8a4d-4b30-85a2-3326ae8094c9 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:44.405877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c67b8-8a4d-4b30-85a2-3326ae8094c9 None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:44.407428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c67b8-8a4d-4b30-85a2-3326ae8094c9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:44.409627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 121/252] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:44 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:44.415895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70a67d-7feb-443b-93c0-8e2fec2d50fc None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:44.546202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70a67d-7feb-443b-93c0-8e2fec2d50fc None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:44.555286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:44.585339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535cca2a-3757-4fb8-9281-acc20d8cc939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:44.586440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cca2a-3757-4fb8-9281-acc20d8cc939 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:44.586919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cca2a-3757-4fb8-9281-acc20d8cc939 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:44.587378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535cca2a-3757-4fb8-9281-acc20d8cc939 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba/users/3d99e94bd2c34f3dbb9b004565d16b88/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:44.589766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535cca2a-3757-4fb8-9281-acc20d8cc939 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a8fffb476e54c20b9662b528c1868ba, user_id=3d99e94bd2c34f3dbb9b004565d16b88, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:44.591444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535cca2a-3757-4fb8-9281-acc20d8cc939 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:44.610064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 132/253] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:16:44 2022] PUT /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba/users/3d99e94bd2c34f3dbb9b004565d16b88/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:16:44.643177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da606244-f360-4051-9f95-93f3f3777890 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:44.643521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da606244-f360-4051-9f95-93f3f3777890 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:44.643837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da606244-f360-4051-9f95-93f3f3777890 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:44.645529 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 122/254] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:44.658238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0bb96-12a8-4483-9a13-f89bba573971 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:44.658538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0bb96-12a8-4483-9a13-f89bba573971 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:44.658782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0bb96-12a8-4483-9a13-f89bba573971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:44.679693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ddf0bb96-12a8-4483-9a13-f89bba573971 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:44.680924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ddf0bb96-12a8-4483-9a13-f89bba573971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:44.767479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 133/255] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:44.982411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a274729-c502-4a04-91aa-c7c4c1eeee9e None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:44.983052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a274729-c502-4a04-91aa-c7c4c1eeee9e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:44.983552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a274729-c502-4a04-91aa-c7c4c1eeee9e None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:44.985490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 123/256] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:44.999400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a542981-3f2c-4dcd-9ba6-6c09d2afd4a1 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:44.999802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a542981-3f2c-4dcd-9ba6-6c09d2afd4a1 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:45.000082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a542981-3f2c-4dcd-9ba6-6c09d2afd4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:45.025516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-1a542981-3f2c-4dcd-9ba6-6c09d2afd4a1 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:45.026796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1a542981-3f2c-4dcd-9ba6-6c09d2afd4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:45.089736 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a274729-c502-4a04-91aa-c7c4c1eeee9e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:45.114189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 134/257] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:45.215341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a274729-c502-4a04-91aa-c7c4c1eeee9e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:45.264958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b04509-a1bc-47fa-a538-8628e033c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:45.265601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b04509-a1bc-47fa-a538-8628e033c53f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:45.265883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b04509-a1bc-47fa-a538-8628e033c53f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:45.266157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b04509-a1bc-47fa-a538-8628e033c53f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:45.282545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b04509-a1bc-47fa-a538-8628e033c53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-d0b04509-a1bc-47fa-a538-8628e033c53f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:45.289383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:45.290228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:45.290992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 124/258] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:16:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:16:45.297824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a542981-3f2c-4dcd-9ba6-6c09d2afd4a1 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:45.335044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:45.435775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a542981-3f2c-4dcd-9ba6-6c09d2afd4a1 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:45.480147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18208b4b-33cb-450b-a13a-02cb68ea5dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:45.481128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18208b4b-33cb-450b-a13a-02cb68ea5dde None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:45.481604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18208b4b-33cb-450b-a13a-02cb68ea5dde None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:45.482062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18208b4b-33cb-450b-a13a-02cb68ea5dde None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:45.482931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18208b4b-33cb-450b-a13a-02cb68ea5dde None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:45.483489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18208b4b-33cb-450b-a13a-02cb68ea5dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:45.485107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18208b4b-33cb-450b-a13a-02cb68ea5dde None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:45.495077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 135/259] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:45.505908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b04509-a1bc-47fa-a538-8628e033c53f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:45.607289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:45.656327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b04509-a1bc-47fa-a538-8628e033c53f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:45.674284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18208b4b-33cb-450b-a13a-02cb68ea5dde None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:45.699312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cbf9e-1be1-43cd-a132-fd44a199f19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:45.699960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cbf9e-1be1-43cd-a132-fd44a199f19b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:45.700240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cbf9e-1be1-43cd-a132-fd44a199f19b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:45.700524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cbf9e-1be1-43cd-a132-fd44a199f19b None admin] PATH_INFO: `/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:45.701730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cbf9e-1be1-43cd-a132-fd44a199f19b None admin] RBAC: Authorizing `identity:get_user(user_id=e654a6c3fda942e2a36c8f8cbd3d0aa6)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:45.703221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cbf9e-1be1-43cd-a132-fd44a199f19b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:45.704849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 125/260] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:45 2022] GET /identity/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:45.711732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cbf9e-1be1-43cd-a132-fd44a199f19b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:45.869272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cbf9e-1be1-43cd-a132-fd44a199f19b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:45.921022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:45.922287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:45.938612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a110b-713f-43ac-b2f9-c5fb6bc12330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:45.939233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a110b-713f-43ac-b2f9-c5fb6bc12330 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:45.939511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a110b-713f-43ac-b2f9-c5fb6bc12330 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:45.939776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a110b-713f-43ac-b2f9-c5fb6bc12330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:45.956333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18208b4b-33cb-450b-a13a-02cb68ea5dde None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:46.032167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a110b-713f-43ac-b2f9-c5fb6bc12330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:46.033763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a110b-713f-43ac-b2f9-c5fb6bc12330 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:46.045022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24a492-410b-49da-9f83-20ccbd92830b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:46.045666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24a492-410b-49da-9f83-20ccbd92830b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:46.045945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24a492-410b-49da-9f83-20ccbd92830b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:46.046220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24a492-410b-49da-9f83-20ccbd92830b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:46.071675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24a492-410b-49da-9f83-20ccbd92830b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-4f24a492-410b-49da-9f83-20ccbd92830b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:46.085393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:46.086304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:46.087073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 136/261] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:16:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:46.093454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24a492-410b-49da-9f83-20ccbd92830b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:46.177662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 126/262] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 00:16:46.183517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a110b-713f-43ac-b2f9-c5fb6bc12330 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:46.215592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:46.312708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24a492-410b-49da-9f83-20ccbd92830b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:46.360691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a110b-713f-43ac-b2f9-c5fb6bc12330 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:46.375088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9e2af-51c4-46ef-b2db-9da744540ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:46.375715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9e2af-51c4-46ef-b2db-9da744540ea9 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:46.375991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9e2af-51c4-46ef-b2db-9da744540ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:46.376262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9e2af-51c4-46ef-b2db-9da744540ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:46.376966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9e2af-51c4-46ef-b2db-9da744540ea9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:46.377269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9e2af-51c4-46ef-b2db-9da744540ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:46.378688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9e2af-51c4-46ef-b2db-9da744540ea9 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:46.392230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 137/263] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:16:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:46.398793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9e2af-51c4-46ef-b2db-9da744540ea9 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:46.426198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3eee12-df5a-44f7-9525-0ffeb66d1a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:46.426854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eee12-df5a-44f7-9525-0ffeb66d1a5a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:46.427463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:46.428630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eee12-df5a-44f7-9525-0ffeb66d1a5a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:46.429015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eee12-df5a-44f7-9525-0ffeb66d1a5a None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:46.433650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3eee12-df5a-44f7-9525-0ffeb66d1a5a None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:46.435230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3eee12-df5a-44f7-9525-0ffeb66d1a5a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:46.437014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 127/264] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:46 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:46.443617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3eee12-df5a-44f7-9525-0ffeb66d1a5a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:46.558471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9e2af-51c4-46ef-b2db-9da744540ea9 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:46.595602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5034e-c304-4b9d-9e5c-65e3b14fc280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:46.596239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5034e-c304-4b9d-9e5c-65e3b14fc280 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:46.596545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5034e-c304-4b9d-9e5c-65e3b14fc280 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:46.596829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5034e-c304-4b9d-9e5c-65e3b14fc280 None admin] PATH_INFO: `/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:46.597981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5034e-c304-4b9d-9e5c-65e3b14fc280 None admin] RBAC: Authorizing `identity:get_user(user_id=e654a6c3fda942e2a36c8f8cbd3d0aa6)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:46.600547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5034e-c304-4b9d-9e5c-65e3b14fc280 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:46.602424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 138/265] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:46 2022] GET /identity/v3/users/e654a6c3fda942e2a36c8f8cbd3d0aa6 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:46.608029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5034e-c304-4b9d-9e5c-65e3b14fc280 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:46.620563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:46.641154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3eee12-df5a-44f7-9525-0ffeb66d1a5a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:46.700874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:46.710619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a78a5b-2545-4fc3-bc85-83446a67f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:46.711237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a78a5b-2545-4fc3-bc85-83446a67f2fb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:46.711519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a78a5b-2545-4fc3-bc85-83446a67f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:46.711794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a78a5b-2545-4fc3-bc85-83446a67f2fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:46.712549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a78a5b-2545-4fc3-bc85-83446a67f2fb None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=9a8fffb476e54c20b9662b528c1868ba, user.id=e654a6c3fda942e2a36c8f8cbd3d0aa6) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:46.713086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a78a5b-2545-4fc3-bc85-83446a67f2fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:46.714607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a78a5b-2545-4fc3-bc85-83446a67f2fb None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:46.726608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5034e-c304-4b9d-9e5c-65e3b14fc280 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:46.728397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 128/266] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:46 2022] GET /identity/v3/role_assignments?user.id=e654a6c3fda942e2a36c8f8cbd3d0aa6&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:46.766788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898816cc-e1d8-4b46-bb38-fb82709f5537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:46.767411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898816cc-e1d8-4b46-bb38-fb82709f5537 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:46.767694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898816cc-e1d8-4b46-bb38-fb82709f5537 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:46.767966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898816cc-e1d8-4b46-bb38-fb82709f5537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:46.846556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898816cc-e1d8-4b46-bb38-fb82709f5537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:46.848060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898816cc-e1d8-4b46-bb38-fb82709f5537 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:46.937329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:46.959316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f502b92-e5e8-476b-88ea-41b824cc5ae4 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:46.959899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f502b92-e5e8-476b-88ea-41b824cc5ae4 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:46.960480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f502b92-e5e8-476b-88ea-41b824cc5ae4 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:46.962147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 129/267] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:46.964523 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 139/268] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 00:16:46.967785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2d06-89fc-48ff-b2fb-21839c254209 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:46.968279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2d06-89fc-48ff-b2fb-21839c254209 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:46.968860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd2d06-89fc-48ff-b2fb-21839c254209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:46.970000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898816cc-e1d8-4b46-bb38-fb82709f5537 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:46.998158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-8fdd2d06-89fc-48ff-b2fb-21839c254209 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:46.999754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8fdd2d06-89fc-48ff-b2fb-21839c254209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:47.081514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898816cc-e1d8-4b46-bb38-fb82709f5537 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:47.121476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced33bb1-1f91-41a2-9459-e7264bc65a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:47.122353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33bb1-1f91-41a2-9459-e7264bc65a7e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.122775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33bb1-1f91-41a2-9459-e7264bc65a7e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.123191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33bb1-1f91-41a2-9459-e7264bc65a7e None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.124477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced33bb1-1f91-41a2-9459-e7264bc65a7e None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:47.126244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced33bb1-1f91-41a2-9459-e7264bc65a7e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:47.128337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 140/269] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:46 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:47.133431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d157ad-62c6-4a04-8058-4aebd4005f38 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.133719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d157ad-62c6-4a04-8058-4aebd4005f38 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.133989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d157ad-62c6-4a04-8058-4aebd4005f38 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.135404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 141/270] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:16:47.135978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d157ad-62c6-4a04-8058-4aebd4005f38 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:47.161061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 130/271] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:47.163389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be832be1-099e-4269-81a8-18f734543298 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.163864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be832be1-099e-4269-81a8-18f734543298 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.164343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be832be1-099e-4269-81a8-18f734543298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.195658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-be832be1-099e-4269-81a8-18f734543298 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:47.198309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-be832be1-099e-4269-81a8-18f734543298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:47.231175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:47.299802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d157ad-62c6-4a04-8058-4aebd4005f38 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:47.310236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 131/272] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:47.312302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b32c4-5377-4cab-9b0e-8d44ce995aa7 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.312754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b32c4-5377-4cab-9b0e-8d44ce995aa7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.313160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b32c4-5377-4cab-9b0e-8d44ce995aa7 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.315030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 132/273] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:47.321106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f8135-54c7-4d06-b710-43753f90c901 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.321559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f8135-54c7-4d06-b710-43753f90c901 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.322015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f8135-54c7-4d06-b710-43753f90c901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.341685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b64f8135-54c7-4d06-b710-43753f90c901 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:47.343243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b64f8135-54c7-4d06-b710-43753f90c901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:47.343444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1dc51-5c82-453e-adb8-e71db074df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:47.344589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1dc51-5c82-453e-adb8-e71db074df1d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.345012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1dc51-5c82-453e-adb8-e71db074df1d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.345427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1dc51-5c82-453e-adb8-e71db074df1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.346293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1dc51-5c82-453e-adb8-e71db074df1d None admin] RBAC: Adding query filter params (role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=9a8fffb476e54c20b9662b528c1868ba, user.id=e654a6c3fda942e2a36c8f8cbd3d0aa6) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:47.346762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1dc51-5c82-453e-adb8-e71db074df1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:47.348366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1dc51-5c82-453e-adb8-e71db074df1d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:47.356713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 142/274] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:47 2022] GET /identity/v3/role_assignments?user.id=e654a6c3fda942e2a36c8f8cbd3d0aa6&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:47.359358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:47.429204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 133/275] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:47.565270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1dc51-5c82-453e-adb8-e71db074df1d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:47.576040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fb494-2935-4a7f-ae86-dd26bbd75ab8 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.579358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fb494-2935-4a7f-ae86-dd26bbd75ab8 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.579641 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fb494-2935-4a7f-ae86-dd26bbd75ab8 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.581310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 134/276] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:47.587376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae56f44-f212-4c1b-8c1a-af145755eccf None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.588244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae56f44-f212-4c1b-8c1a-af145755eccf None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.588752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae56f44-f212-4c1b-8c1a-af145755eccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.616117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-dae56f44-f212-4c1b-8c1a-af145755eccf None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:47.618522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-dae56f44-f212-4c1b-8c1a-af145755eccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:47.712108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 135/277] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:47.713670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae56f44-f212-4c1b-8c1a-af145755eccf None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:47.724371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1dc51-5c82-453e-adb8-e71db074df1d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:47.740743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:47.771911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23638e03-b804-4fc9-ae2f-146c772ccfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:47.772745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23638e03-b804-4fc9-ae2f-146c772ccfd3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.773018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23638e03-b804-4fc9-ae2f-146c772ccfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.773282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23638e03-b804-4fc9-ae2f-146c772ccfd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.783770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23638e03-b804-4fc9-ae2f-146c772ccfd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-23638e03-b804-4fc9-ae2f-146c772ccfd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:47.790135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:47.791065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:47.791824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:47.794294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 143/278] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:47.795083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23638e03-b804-4fc9-ae2f-146c772ccfd3 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:47.812423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:47.840210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae56f44-f212-4c1b-8c1a-af145755eccf None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:47.879150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2ae13-83e0-4fc6-9c82-612c54c1214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:47.880020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2ae13-83e0-4fc6-9c82-612c54c1214c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.880591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2ae13-83e0-4fc6-9c82-612c54c1214c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.881000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2ae13-83e0-4fc6-9c82-612c54c1214c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.882232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2ae13-83e0-4fc6-9c82-612c54c1214c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:47.883811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2ae13-83e0-4fc6-9c82-612c54c1214c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:47.885541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 136/279] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:47.886440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2ae13-83e0-4fc6-9c82-612c54c1214c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:47.914500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23638e03-b804-4fc9-ae2f-146c772ccfd3 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:47.949012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85f38b-e58b-4bf2-b31e-3731dfca1d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:47.949832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85f38b-e58b-4bf2-b31e-3731dfca1d53 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:47.950270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85f38b-e58b-4bf2-b31e-3731dfca1d53 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:47.950684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85f38b-e58b-4bf2-b31e-3731dfca1d53 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:47.962379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b85f38b-e58b-4bf2-b31e-3731dfca1d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-8b85f38b-e58b-4bf2-b31e-3731dfca1d53 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:47.967779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:47.968707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:47.969468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:47.971293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 144/280] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:47.972199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85f38b-e58b-4bf2-b31e-3731dfca1d53 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:47.997236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2ae13-83e0-4fc6-9c82-612c54c1214c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.004307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:48.031378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468969e7-221d-4bd9-a9a7-af75efa39eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.031977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468969e7-221d-4bd9-a9a7-af75efa39eee None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.032253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468969e7-221d-4bd9-a9a7-af75efa39eee None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.032543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468969e7-221d-4bd9-a9a7-af75efa39eee None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.033197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468969e7-221d-4bd9-a9a7-af75efa39eee None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:48.033493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468969e7-221d-4bd9-a9a7-af75efa39eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.034870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468969e7-221d-4bd9-a9a7-af75efa39eee None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.042764 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 137/281] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:48.043541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468969e7-221d-4bd9-a9a7-af75efa39eee None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:48.048456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:48.074619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85f38b-e58b-4bf2-b31e-3731dfca1d53 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.107640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93d824-05f8-453c-91ea-24b1f101d8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.108168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93d824-05f8-453c-91ea-24b1f101d8bf None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.108443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93d824-05f8-453c-91ea-24b1f101d8bf None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.108738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93d824-05f8-453c-91ea-24b1f101d8bf None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.109374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93d824-05f8-453c-91ea-24b1f101d8bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.110835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93d824-05f8-453c-91ea-24b1f101d8bf None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.132073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 145/282] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:16:47 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:16:48.132947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93d824-05f8-453c-91ea-24b1f101d8bf None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:48.146499 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468969e7-221d-4bd9-a9a7-af75efa39eee None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.183581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140a63f-0717-472b-884c-15904bceaf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.184202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140a63f-0717-472b-884c-15904bceaf84 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.184481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140a63f-0717-472b-884c-15904bceaf84 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.184778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140a63f-0717-472b-884c-15904bceaf84 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.185460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140a63f-0717-472b-884c-15904bceaf84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:48.185762 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140a63f-0717-472b-884c-15904bceaf84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.187191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140a63f-0717-472b-884c-15904bceaf84 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.201652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 138/283] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:48.201652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140a63f-0717-472b-884c-15904bceaf84 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:48.247036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:48.269421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93d824-05f8-453c-91ea-24b1f101d8bf None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.274509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:48.316059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc4386-5bdc-4bf5-b468-f10d28aaeaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.316421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140a63f-0717-472b-884c-15904bceaf84 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.319911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4386-5bdc-4bf5-b468-f10d28aaeaf9 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.319911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4386-5bdc-4bf5-b468-f10d28aaeaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.319911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4386-5bdc-4bf5-b468-f10d28aaeaf9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.330852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc4386-5bdc-4bf5-b468-f10d28aaeaf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-efcc4386-5bdc-4bf5-b468-f10d28aaeaf9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:48.337932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:48.338839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 146/284] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:48.339633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc4386-5bdc-4bf5-b468-f10d28aaeaf9 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:48.366711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850f61e-b858-445d-bd61-bffbdf9328b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.367274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850f61e-b858-445d-bd61-bffbdf9328b2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.367550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850f61e-b858-445d-bd61-bffbdf9328b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.370134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850f61e-b858-445d-bd61-bffbdf9328b2 None admin] PATH_INFO: `/v3/users/7840473cb1f84f61bc8ae72269e62999` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.385743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850f61e-b858-445d-bd61-bffbdf9328b2 None admin] RBAC: Authorizing `identity:get_user(user_id=7840473cb1f84f61bc8ae72269e62999)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.387208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850f61e-b858-445d-bd61-bffbdf9328b2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.389089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 139/285] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:48 2022] GET /identity/v3/users/7840473cb1f84f61bc8ae72269e62999 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:48.389684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850f61e-b858-445d-bd61-bffbdf9328b2 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:48.490274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc4386-5bdc-4bf5-b468-f10d28aaeaf9 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.522543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850f61e-b858-445d-bd61-bffbdf9328b2 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.527824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcf1f8-82f2-480b-928e-63be9f0d927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.528457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcf1f8-82f2-480b-928e-63be9f0d927b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.528764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcf1f8-82f2-480b-928e-63be9f0d927b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.529036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcf1f8-82f2-480b-928e-63be9f0d927b None admin] PATH_INFO: `/v3/users/6b23de1a521e481dab26f9905b088761` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.530185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcf1f8-82f2-480b-928e-63be9f0d927b None admin] RBAC: Authorizing `identity:get_user(user_id=6b23de1a521e481dab26f9905b088761)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.531698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcf1f8-82f2-480b-928e-63be9f0d927b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.533463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 147/286] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:48 2022] GET /identity/v3/users/6b23de1a521e481dab26f9905b088761 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:48.534325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcf1f8-82f2-480b-928e-63be9f0d927b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:48.550237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:48.553407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:48.563337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb664d63-a309-431b-b2d6-f5593b65c3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.564046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb664d63-a309-431b-b2d6-f5593b65c3bf None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.564430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb664d63-a309-431b-b2d6-f5593b65c3bf None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.564832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb664d63-a309-431b-b2d6-f5593b65c3bf None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.565600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb664d63-a309-431b-b2d6-f5593b65c3bf None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:48.566014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb664d63-a309-431b-b2d6-f5593b65c3bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.567495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb664d63-a309-431b-b2d6-f5593b65c3bf None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.577786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 140/287] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:48.579636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb664d63-a309-431b-b2d6-f5593b65c3bf None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:48.648071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcf1f8-82f2-480b-928e-63be9f0d927b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.690237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588db910-a2e7-471e-b289-b542658c2f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.691176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588db910-a2e7-471e-b289-b542658c2f70 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.691631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588db910-a2e7-471e-b289-b542658c2f70 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.692091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588db910-a2e7-471e-b289-b542658c2f70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.692303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb664d63-a309-431b-b2d6-f5593b65c3bf None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.693332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588db910-a2e7-471e-b289-b542658c2f70 None admin] RBAC: Adding query filter params (role.id=df2ecb74b9fe42089e380b16de5ad950, user.id=7840473cb1f84f61bc8ae72269e62999, scope.system=all) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:48.693838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588db910-a2e7-471e-b289-b542658c2f70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.695678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588db910-a2e7-471e-b289-b542658c2f70 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.701301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 148/288] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 1 00:16:48 2022] GET /identity/v3/role_assignments?user.id=7840473cb1f84f61bc8ae72269e62999&scope.system=all&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:48.703489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588db910-a2e7-471e-b289-b542658c2f70 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:48.735631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ec863-cfde-4d21-b123-cd0c03f48641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.736394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ec863-cfde-4d21-b123-cd0c03f48641 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.736618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ec863-cfde-4d21-b123-cd0c03f48641 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.736894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ec863-cfde-4d21-b123-cd0c03f48641 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.738068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ec863-cfde-4d21-b123-cd0c03f48641 None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.740159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ec863-cfde-4d21-b123-cd0c03f48641 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.741971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 141/289] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:48 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:48.747854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ec863-cfde-4d21-b123-cd0c03f48641 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:48.812913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:48.868457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ec863-cfde-4d21-b123-cd0c03f48641 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.892325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:48.916367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b80118-8fad-4042-b3a7-ecd58e1720b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.917431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b80118-8fad-4042-b3a7-ecd58e1720b8 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.917719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b80118-8fad-4042-b3a7-ecd58e1720b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.917996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b80118-8fad-4042-b3a7-ecd58e1720b8 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba/users/6b23de1a521e481dab26f9905b088761/roles/a6588e7acf534502a191a0bca0d46c39` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.920516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b80118-8fad-4042-b3a7-ecd58e1720b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a8fffb476e54c20b9662b528c1868ba, user_id=6b23de1a521e481dab26f9905b088761, role_id=a6588e7acf534502a191a0bca0d46c39)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.922705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b80118-8fad-4042-b3a7-ecd58e1720b8 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.924263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588db910-a2e7-471e-b289-b542658c2f70 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:48.938318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 142/290] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:16:48 2022] PUT /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba/users/6b23de1a521e481dab26f9905b088761/roles/a6588e7acf534502a191a0bca0d46c39 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:16:48.989282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0c18a-2912-4d0a-800e-3dd5ab253a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:48.991061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0c18a-2912-4d0a-800e-3dd5ab253a35 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:48.991589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0c18a-2912-4d0a-800e-3dd5ab253a35 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:48.992062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0c18a-2912-4d0a-800e-3dd5ab253a35 None admin] PATH_INFO: `/v3/users/3d99e94bd2c34f3dbb9b004565d16b88` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:48.993678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0c18a-2912-4d0a-800e-3dd5ab253a35 None admin] RBAC: Authorizing `identity:get_user(user_id=3d99e94bd2c34f3dbb9b004565d16b88)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:48.995419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0c18a-2912-4d0a-800e-3dd5ab253a35 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:48.997494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 149/291] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:48 2022] GET /identity/v3/users/3d99e94bd2c34f3dbb9b004565d16b88 => generated 297 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:49.003571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b80118-8fad-4042-b3a7-ecd58e1720b8 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:49.143175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b80118-8fad-4042-b3a7-ecd58e1720b8 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:49.159986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:49.189174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfaecb-0436-4e26-b272-bce806460937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:49.189716 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfaecb-0436-4e26-b272-bce806460937 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:49.189993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfaecb-0436-4e26-b272-bce806460937 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:49.190263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfaecb-0436-4e26-b272-bce806460937 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:49.276095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfaecb-0436-4e26-b272-bce806460937 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:49.277608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfaecb-0436-4e26-b272-bce806460937 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:49.402100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 143/292] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:49.407364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0c18a-2912-4d0a-800e-3dd5ab253a35 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:49.547908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0c18a-2912-4d0a-800e-3dd5ab253a35 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:49.588671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03334cc5-d05c-4bfc-bcd9-9a552a2a9d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:49.589315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03334cc5-d05c-4bfc-bcd9-9a552a2a9d4b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:49.589595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03334cc5-d05c-4bfc-bcd9-9a552a2a9d4b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:49.589871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03334cc5-d05c-4bfc-bcd9-9a552a2a9d4b None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:49.591039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03334cc5-d05c-4bfc-bcd9-9a552a2a9d4b None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:49.592593 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03334cc5-d05c-4bfc-bcd9-9a552a2a9d4b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:49.594233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 150/293] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:49 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:49.600986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfaecb-0436-4e26-b272-bce806460937 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:49.633643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:49.743188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfaecb-0436-4e26-b272-bce806460937 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:49.784741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1436f-0a82-499a-96c8-632dc83e764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:49.785346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1436f-0a82-499a-96c8-632dc83e764d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:49.785623 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1436f-0a82-499a-96c8-632dc83e764d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:49.785893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1436f-0a82-499a-96c8-632dc83e764d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:49.787136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1436f-0a82-499a-96c8-632dc83e764d None admin] RBAC: Adding query filter params (role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=9a8fffb476e54c20b9662b528c1868ba, user.id=3d99e94bd2c34f3dbb9b004565d16b88) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:49.787437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1436f-0a82-499a-96c8-632dc83e764d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:49.788926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1436f-0a82-499a-96c8-632dc83e764d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:49.794997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 144/294] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:49 2022] GET /identity/v3/role_assignments?user.id=3d99e94bd2c34f3dbb9b004565d16b88&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:50.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803cc0-acf0-4c2d-b780-f83f51281457 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:50.491490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803cc0-acf0-4c2d-b780-f83f51281457 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:50.491668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803cc0-acf0-4c2d-b780-f83f51281457 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:50.493306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 151/295] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:50.498118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cd26b-2996-44d7-8c69-265f6dd97b34 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:50.498397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cd26b-2996-44d7-8c69-265f6dd97b34 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:50.498673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cd26b-2996-44d7-8c69-265f6dd97b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:50.517441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0b3cd26b-2996-44d7-8c69-265f6dd97b34 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:50.518617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0b3cd26b-2996-44d7-8c69-265f6dd97b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:50.592513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 145/296] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:50.640321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c7337-ec8a-430a-9004-b9ff965c74a5 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:50.640850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c7337-ec8a-430a-9004-b9ff965c74a5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:50.640922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c7337-ec8a-430a-9004-b9ff965c74a5 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:50.642225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 152/297] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:50.647962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e3ae8-e102-41a2-a80d-bbbf822690ae None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:50.648249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e3ae8-e102-41a2-a80d-bbbf822690ae None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:50.648547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e3ae8-e102-41a2-a80d-bbbf822690ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:50.666829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ad9e3ae8-e102-41a2-a80d-bbbf822690ae None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:50.668083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ad9e3ae8-e102-41a2-a80d-bbbf822690ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:50.670126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:50.739928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 146/298] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:50.872514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c7337-ec8a-430a-9004-b9ff965c74a5 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:50.889952 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:50.992989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c7337-ec8a-430a-9004-b9ff965c74a5 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:51.019392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e3ae8-e102-41a2-a80d-bbbf822690ae None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:51.031826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37858a-fbc3-4247-bc5e-80fcb99c5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:51.032975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37858a-fbc3-4247-bc5e-80fcb99c5723 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.033753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37858a-fbc3-4247-bc5e-80fcb99c5723 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.034565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37858a-fbc3-4247-bc5e-80fcb99c5723 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.036637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37858a-fbc3-4247-bc5e-80fcb99c5723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:51.039284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37858a-fbc3-4247-bc5e-80fcb99c5723 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:51.042223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 153/299] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:51.048091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37858a-fbc3-4247-bc5e-80fcb99c5723 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:51.133259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e3ae8-e102-41a2-a80d-bbbf822690ae None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:51.155771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:51.180675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37858a-fbc3-4247-bc5e-80fcb99c5723 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:51.182582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bff18-0b54-4344-90d5-636dc764fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:51.183227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bff18-0b54-4344-90d5-636dc764fa8b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.183497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bff18-0b54-4344-90d5-636dc764fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.183773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bff18-0b54-4344-90d5-636dc764fa8b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.190817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:51.194845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527bff18-0b54-4344-90d5-636dc764fa8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-527bff18-0b54-4344-90d5-636dc764fa8b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:51.199918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:51.200917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:51.201713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 147/300] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:16:51.202541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d6369-694e-4fbf-8ca9-b42d5ef746d2 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.202541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d6369-694e-4fbf-8ca9-b42d5ef746d2 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.202722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d6369-694e-4fbf-8ca9-b42d5ef746d2 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.204454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 148/301] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:51.206570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a62e80-058b-4965-8e90-9e99cdb9dd5f None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.206874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a62e80-058b-4965-8e90-9e99cdb9dd5f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.207142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a62e80-058b-4965-8e90-9e99cdb9dd5f None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.212336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 149/302] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:51.212967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a62e80-058b-4965-8e90-9e99cdb9dd5f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:51.235232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc50b34-b281-40a4-bae7-ef60cc9ecb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:51.235849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc50b34-b281-40a4-bae7-ef60cc9ecb26 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.236130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc50b34-b281-40a4-bae7-ef60cc9ecb26 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.236478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc50b34-b281-40a4-bae7-ef60cc9ecb26 None admin] PATH_INFO: `/v3/groups` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.237066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc50b34-b281-40a4-bae7-ef60cc9ecb26 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:51.238986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc50b34-b281-40a4-bae7-ef60cc9ecb26 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:51.258500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 154/303] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 1 00:16:51 2022] POST /identity/v3/groups => generated 229 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:16:51.259908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944e1a7-6fbd-46f0-8940-b094c490f46b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.260190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944e1a7-6fbd-46f0-8940-b094c490f46b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.260753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944e1a7-6fbd-46f0-8940-b094c490f46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.282503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0944e1a7-6fbd-46f0-8940-b094c490f46b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:51.283789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0944e1a7-6fbd-46f0-8940-b094c490f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:51.366744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a62e80-058b-4965-8e90-9e99cdb9dd5f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:51.376635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 155/304] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:51.376635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e403b-a944-45c6-895a-13388b7d9089 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.376635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e403b-a944-45c6-895a-13388b7d9089 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.376635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e403b-a944-45c6-895a-13388b7d9089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.392590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-fc0e403b-a944-45c6-895a-13388b7d9089 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:51.393878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fc0e403b-a944-45c6-895a-13388b7d9089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:51.436942 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264df61-6093-400b-8bc8-08f6d83bbbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:51.437679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264df61-6093-400b-8bc8-08f6d83bbbdc None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.438060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264df61-6093-400b-8bc8-08f6d83bbbdc None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.438442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264df61-6093-400b-8bc8-08f6d83bbbdc None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.439239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264df61-6093-400b-8bc8-08f6d83bbbdc None admin] RBAC: Adding query filter params (name=reader) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:51.439547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264df61-6093-400b-8bc8-08f6d83bbbdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:51.441170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264df61-6093-400b-8bc8-08f6d83bbbdc None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:51.458678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 150/305] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:51.467319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264df61-6093-400b-8bc8-08f6d83bbbdc None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:51.486363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 156/306] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:51.540475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:51.584675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264df61-6093-400b-8bc8-08f6d83bbbdc None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:51.621510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc02ead-c024-47fa-a523-6decd257dc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:51.622830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02ead-c024-47fa-a523-6decd257dc6a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.623287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02ead-c024-47fa-a523-6decd257dc6a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.623735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc02ead-c024-47fa-a523-6decd257dc6a None admin] PATH_INFO: `/v3/users/7fb6da991055455e921ce4a3399e73b5` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.639604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc02ead-c024-47fa-a523-6decd257dc6a None admin] RBAC: Authorizing `identity:get_user(user_id=7fb6da991055455e921ce4a3399e73b5)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:51.641553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc02ead-c024-47fa-a523-6decd257dc6a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:51.643513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 151/307] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:51 2022] GET /identity/v3/users/7fb6da991055455e921ce4a3399e73b5 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:51.652904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc02ead-c024-47fa-a523-6decd257dc6a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:51.653060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e403b-a944-45c6-895a-13388b7d9089 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:51.656870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:51.779714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc02ead-c024-47fa-a523-6decd257dc6a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:51.790767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e403b-a944-45c6-895a-13388b7d9089 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:51.799449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:51.820477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b039a8-d474-487c-8a55-17b422914e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:51.821156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b039a8-d474-487c-8a55-17b422914e98 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.821418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b039a8-d474-487c-8a55-17b422914e98 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.821690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b039a8-d474-487c-8a55-17b422914e98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.829752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aacb78-810d-44d9-a45f-5d8871c739a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:51.830634 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aacb78-810d-44d9-a45f-5d8871c739a4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:51.830912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aacb78-810d-44d9-a45f-5d8871c739a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:51.831256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aacb78-810d-44d9-a45f-5d8871c739a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:51.832064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aacb78-810d-44d9-a45f-5d8871c739a4 None admin] RBAC: Adding query filter params (role.id=a6588e7acf534502a191a0bca0d46c39, user.id=7fb6da991055455e921ce4a3399e73b5, scope.system=all) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:51.832535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aacb78-810d-44d9-a45f-5d8871c739a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:51.834195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aacb78-810d-44d9-a45f-5d8871c739a4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:51.835061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b039a8-d474-487c-8a55-17b422914e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:51.839569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 157/308] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 1 00:16:51 2022] GET /identity/v3/role_assignments?user.id=7fb6da991055455e921ce4a3399e73b5&scope.system=all&role.id=a6588e7acf534502a191a0bca0d46c39 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:51.840381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aacb78-810d-44d9-a45f-5d8871c739a4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-24b039a8-d474-487c-8a55-17b422914e98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:51.841212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:51.842085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:51.842866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:51.843445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 152/309] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:16:51.851986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b039a8-d474-487c-8a55-17b422914e98 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:51.952013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:51.989193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b039a8-d474-487c-8a55-17b422914e98 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:52.003765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aacb78-810d-44d9-a45f-5d8871c739a4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:52.035471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25857853-095f-4b62-8d8f-f8be02e3d5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:52.036068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25857853-095f-4b62-8d8f-f8be02e3d5bc None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:52.036922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25857853-095f-4b62-8d8f-f8be02e3d5bc None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:52.037402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25857853-095f-4b62-8d8f-f8be02e3d5bc None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:52.038558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25857853-095f-4b62-8d8f-f8be02e3d5bc None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:52.038908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25857853-095f-4b62-8d8f-f8be02e3d5bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:52.040475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25857853-095f-4b62-8d8f-f8be02e3d5bc None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:52.049929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 153/310] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:52.055172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205b449-e0d2-472d-afa5-6e99fdcedf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:52.056022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205b449-e0d2-472d-afa5-6e99fdcedf96 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:52.056446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205b449-e0d2-472d-afa5-6e99fdcedf96 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:52.056758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25857853-095f-4b62-8d8f-f8be02e3d5bc None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:52.057108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205b449-e0d2-472d-afa5-6e99fdcedf96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:52.073479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6205b449-e0d2-472d-afa5-6e99fdcedf96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-6205b449-e0d2-472d-afa5-6e99fdcedf96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:52.080549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:52.081626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:52.082609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:52.085405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 158/311] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:52.091137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205b449-e0d2-472d-afa5-6e99fdcedf96 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:52.107784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:52.169787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25857853-095f-4b62-8d8f-f8be02e3d5bc None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:52.204897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205b449-e0d2-472d-afa5-6e99fdcedf96 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:52.206029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a54b7e-cc77-455c-bace-93fb0b79674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:52.206882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54b7e-cc77-455c-bace-93fb0b79674a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:52.207389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54b7e-cc77-455c-bace-93fb0b79674a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:52.207889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54b7e-cc77-455c-bace-93fb0b79674a None admin] PATH_INFO: `/v3/users/7840473cb1f84f61bc8ae72269e62999` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:52.211111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a54b7e-cc77-455c-bace-93fb0b79674a None admin] RBAC: Authorizing `identity:get_user(user_id=7840473cb1f84f61bc8ae72269e62999)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:52.212197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a54b7e-cc77-455c-bace-93fb0b79674a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:52.214603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 154/312] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:52 2022] GET /identity/v3/users/7840473cb1f84f61bc8ae72269e62999 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:52.220534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a54b7e-cc77-455c-bace-93fb0b79674a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:52.251616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:52.263445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eed912-a9ce-4f0a-8a8b-d80f456e77aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:52.264329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eed912-a9ce-4f0a-8a8b-d80f456e77aa None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:52.264856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eed912-a9ce-4f0a-8a8b-d80f456e77aa None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:52.265328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eed912-a9ce-4f0a-8a8b-d80f456e77aa None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:52.266302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eed912-a9ce-4f0a-8a8b-d80f456e77aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:52.266776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eed912-a9ce-4f0a-8a8b-d80f456e77aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:52.268434 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eed912-a9ce-4f0a-8a8b-d80f456e77aa None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:52.283706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 159/313] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:52.284926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eed912-a9ce-4f0a-8a8b-d80f456e77aa None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:52.331858 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a54b7e-cc77-455c-bace-93fb0b79674a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:52.342495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:52.372532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6be34c-adbd-407b-bca1-b0be8541b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:52.372970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6be34c-adbd-407b-bca1-b0be8541b097 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:52.373251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6be34c-adbd-407b-bca1-b0be8541b097 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:52.373528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6be34c-adbd-407b-bca1-b0be8541b097 None admin] PATH_INFO: `/v3/system/users/7840473cb1f84f61bc8ae72269e62999/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:52.374959 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6be34c-adbd-407b-bca1-b0be8541b097 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7840473cb1f84f61bc8ae72269e62999, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:52.376462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6be34c-adbd-407b-bca1-b0be8541b097 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:52.387450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 155/314] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 1 00:16:52 2022] PUT /identity/v3/system/users/7840473cb1f84f61bc8ae72269e62999/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:16:52.408910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eed912-a9ce-4f0a-8a8b-d80f456e77aa None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:52.455042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96049d17-8567-492b-9d52-4b464818578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:52.455594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96049d17-8567-492b-9d52-4b464818578f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:52.455873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96049d17-8567-492b-9d52-4b464818578f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:52.456144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96049d17-8567-492b-9d52-4b464818578f None admin] PATH_INFO: `/v3/users/6b23de1a521e481dab26f9905b088761` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:52.457379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96049d17-8567-492b-9d52-4b464818578f None admin] RBAC: Authorizing `identity:get_user(user_id=6b23de1a521e481dab26f9905b088761)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:52.459414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96049d17-8567-492b-9d52-4b464818578f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:52.461109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 160/315] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:52 2022] GET /identity/v3/users/6b23de1a521e481dab26f9905b088761 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:52.467041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6be34c-adbd-407b-bca1-b0be8541b097 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:52.587444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6be34c-adbd-407b-bca1-b0be8541b097 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:52.624275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be45341-bd2f-406c-a823-b63916a309d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:52.624935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be45341-bd2f-406c-a823-b63916a309d5 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:52.625216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be45341-bd2f-406c-a823-b63916a309d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:52.625488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be45341-bd2f-406c-a823-b63916a309d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:52.669238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:52.700528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be45341-bd2f-406c-a823-b63916a309d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:52.702070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be45341-bd2f-406c-a823-b63916a309d5 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:52.814280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 156/316] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:52.820358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96049d17-8567-492b-9d52-4b464818578f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:52.882457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:52.929399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96049d17-8567-492b-9d52-4b464818578f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:52.965982 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d1e73-b0e5-462a-a210-e6c89798ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:52.967179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d1e73-b0e5-462a-a210-e6c89798ec77 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:52.967888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d1e73-b0e5-462a-a210-e6c89798ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:52.968365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d1e73-b0e5-462a-a210-e6c89798ec77 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:52.969764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d1e73-b0e5-462a-a210-e6c89798ec77 None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:52.971473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d1e73-b0e5-462a-a210-e6c89798ec77 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:52.973498 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 161/317] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:52 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:52.980467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be45341-bd2f-406c-a823-b63916a309d5 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:53.056758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:53.090379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be45341-bd2f-406c-a823-b63916a309d5 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:53.127883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87143a-e90e-4ed2-a528-4c3555dced39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:53.128891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87143a-e90e-4ed2-a528-4c3555dced39 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:53.129395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87143a-e90e-4ed2-a528-4c3555dced39 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:53.129924 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87143a-e90e-4ed2-a528-4c3555dced39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:53.130956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87143a-e90e-4ed2-a528-4c3555dced39 None admin] RBAC: Adding query filter params (role.id=a6588e7acf534502a191a0bca0d46c39, scope.project.id=9a8fffb476e54c20b9662b528c1868ba, user.id=6b23de1a521e481dab26f9905b088761) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:53.131492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87143a-e90e-4ed2-a528-4c3555dced39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:53.133160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87143a-e90e-4ed2-a528-4c3555dced39 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:53.139633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 157/318] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:16:52 2022] GET /identity/v3/role_assignments?user.id=6b23de1a521e481dab26f9905b088761&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=a6588e7acf534502a191a0bca0d46c39 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:53.393109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc93b1-f72b-4a51-a01e-675a14c7e046 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:53.393378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc93b1-f72b-4a51-a01e-675a14c7e046 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:53.393610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc93b1-f72b-4a51-a01e-675a14c7e046 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:53.395068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 162/319] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:53.400606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37bd02-0023-46ad-a52e-69caed3418b0 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:53.400893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37bd02-0023-46ad-a52e-69caed3418b0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:53.401165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37bd02-0023-46ad-a52e-69caed3418b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:53.419316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-7d37bd02-0023-46ad-a52e-69caed3418b0 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:53.420510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7d37bd02-0023-46ad-a52e-69caed3418b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:53.487642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 158/320] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:53.758950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc93b1-f72b-4a51-a01e-675a14c7e046 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:53.826542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f6d22-ac47-4f9d-a624-9102dc7afb9d None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:53.826838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f6d22-ac47-4f9d-a624-9102dc7afb9d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:53.827115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f6d22-ac47-4f9d-a624-9102dc7afb9d None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:53.828644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 159/321] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:16:53.833728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655e076-0ff3-4adf-b453-d2031a0f22f6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:53.834010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655e076-0ff3-4adf-b453-d2031a0f22f6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:53.834280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655e076-0ff3-4adf-b453-d2031a0f22f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:53.852784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1655e076-0ff3-4adf-b453-d2031a0f22f6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:53.853974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1655e076-0ff3-4adf-b453-d2031a0f22f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:53.863147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc93b1-f72b-4a51-a01e-675a14c7e046 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:53.889456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:53.903249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f161e-4b1e-4c1d-85c6-2a27170aa69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:53.903972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f161e-4b1e-4c1d-85c6-2a27170aa69f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:53.904318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f161e-4b1e-4c1d-85c6-2a27170aa69f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:53.904692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f161e-4b1e-4c1d-85c6-2a27170aa69f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:53.905850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f161e-4b1e-4c1d-85c6-2a27170aa69f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:53.907382 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f161e-4b1e-4c1d-85c6-2a27170aa69f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:53.909255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 163/322] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:16:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:53.915470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f161e-4b1e-4c1d-85c6-2a27170aa69f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:53.922996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:53.932536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 160/323] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:54.018485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f161e-4b1e-4c1d-85c6-2a27170aa69f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:54.052366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3305e6e-806b-4d55-b46c-8b53594be6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:54.053208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3305e6e-806b-4d55-b46c-8b53594be6e2 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:54.053623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3305e6e-806b-4d55-b46c-8b53594be6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:54.054028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3305e6e-806b-4d55-b46c-8b53594be6e2 None admin] PATH_INFO: `/v3/groups` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:54.054780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3305e6e-806b-4d55-b46c-8b53594be6e2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:54.056376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3305e6e-806b-4d55-b46c-8b53594be6e2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:54.073777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 164/324] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 1 00:16:53 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:16:54.210493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655e076-0ff3-4adf-b453-d2031a0f22f6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:54.316905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655e076-0ff3-4adf-b453-d2031a0f22f6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:54.354290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cf26e-fab0-4e66-ba40-453e64269b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:54.354927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cf26e-fab0-4e66-ba40-453e64269b05 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:54.355258 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cf26e-fab0-4e66-ba40-453e64269b05 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:54.355471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cf26e-fab0-4e66-ba40-453e64269b05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:54.367065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85cf26e-fab0-4e66-ba40-453e64269b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-e85cf26e-fab0-4e66-ba40-453e64269b05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:54.371977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:54.373124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:54.374026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 161/325] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:54.381119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3305e6e-806b-4d55-b46c-8b53594be6e2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:54.441682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:54.450915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac6681-dc23-4d80-b717-e31c69e983ee None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:54.451304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac6681-dc23-4d80-b717-e31c69e983ee None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:54.451579 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac6681-dc23-4d80-b717-e31c69e983ee None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:54.453558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 162/326] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:16:54.459913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cb918-9d81-4df6-b964-dd8cc46c035c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:54.460196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cb918-9d81-4df6-b964-dd8cc46c035c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:54.460470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cb918-9d81-4df6-b964-dd8cc46c035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:54.481093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-df2cb918-9d81-4df6-b964-dd8cc46c035c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:54.482210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-df2cb918-9d81-4df6-b964-dd8cc46c035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:54.497647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3305e6e-806b-4d55-b46c-8b53594be6e2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:54.549346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcd46d-2512-44f0-bcde-660d1b15c085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:54.549987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcd46d-2512-44f0-bcde-660d1b15c085 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:54.550267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcd46d-2512-44f0-bcde-660d1b15c085 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:54.550959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcd46d-2512-44f0-bcde-660d1b15c085 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:54.552441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcd46d-2512-44f0-bcde-660d1b15c085 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:54.552844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcd46d-2512-44f0-bcde-660d1b15c085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:54.554541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcd46d-2512-44f0-bcde-660d1b15c085 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:54.560015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:54.568618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 165/327] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:54.571829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcd46d-2512-44f0-bcde-660d1b15c085 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:54.599379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 163/328] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:54.690973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcd46d-2512-44f0-bcde-660d1b15c085 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:54.705849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:54.734149 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca27b25-3866-40fc-9ca9-e1787998dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:54.735032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca27b25-3866-40fc-9ca9-e1787998dc52 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:54.735459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca27b25-3866-40fc-9ca9-e1787998dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:54.735899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca27b25-3866-40fc-9ca9-e1787998dc52 None admin] PATH_INFO: `/v3/users/7fb6da991055455e921ce4a3399e73b5` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:54.737245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca27b25-3866-40fc-9ca9-e1787998dc52 None admin] RBAC: Authorizing `identity:get_user(user_id=7fb6da991055455e921ce4a3399e73b5)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:54.738945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca27b25-3866-40fc-9ca9-e1787998dc52 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:54.742300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 166/329] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:54 2022] GET /identity/v3/users/7fb6da991055455e921ce4a3399e73b5 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:54.749718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cb918-9d81-4df6-b964-dd8cc46c035c None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:54.879429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cb918-9d81-4df6-b964-dd8cc46c035c None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:54.921064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a0f22-c890-49a5-989b-6f2fef43490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:54.921965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a0f22-c890-49a5-989b-6f2fef43490c None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:54.922439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a0f22-c890-49a5-989b-6f2fef43490c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:54.922841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a0f22-c890-49a5-989b-6f2fef43490c None admin] PATH_INFO: `/v3/system/users/7fb6da991055455e921ce4a3399e73b5/roles/a6588e7acf534502a191a0bca0d46c39` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:54.924416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a0f22-c890-49a5-989b-6f2fef43490c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7fb6da991055455e921ce4a3399e73b5, role_id=a6588e7acf534502a191a0bca0d46c39)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:54.925957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a0f22-c890-49a5-989b-6f2fef43490c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:54.937704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 164/330] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 1 00:16:54 2022] PUT /identity/v3/system/users/7fb6da991055455e921ce4a3399e73b5/roles/a6588e7acf534502a191a0bca0d46c39 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:16:54.948614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca27b25-3866-40fc-9ca9-e1787998dc52 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:55.068479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca27b25-3866-40fc-9ca9-e1787998dc52 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:55.112673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881e1d7-fbc9-4157-a776-4fd2857d9df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:55.113644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881e1d7-fbc9-4157-a776-4fd2857d9df2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:55.113920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881e1d7-fbc9-4157-a776-4fd2857d9df2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:55.114268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881e1d7-fbc9-4157-a776-4fd2857d9df2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:55.126659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881e1d7-fbc9-4157-a776-4fd2857d9df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-d881e1d7-fbc9-4157-a776-4fd2857d9df2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:55.132924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:55.133960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:55.134879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:55.137164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 167/331] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:55.143521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a0f22-c890-49a5-989b-6f2fef43490c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:55.219365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:55.296317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a0f22-c890-49a5-989b-6f2fef43490c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:55.343061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab68f4d-b917-446a-ba11-071a4d34e85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:55.343850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab68f4d-b917-446a-ba11-071a4d34e85d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:55.344230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab68f4d-b917-446a-ba11-071a4d34e85d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:55.344624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab68f4d-b917-446a-ba11-071a4d34e85d None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:55.345404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab68f4d-b917-446a-ba11-071a4d34e85d None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:55.345806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab68f4d-b917-446a-ba11-071a4d34e85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:55.347292 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab68f4d-b917-446a-ba11-071a4d34e85d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:55.360104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 165/332] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:55.366243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881e1d7-fbc9-4157-a776-4fd2857d9df2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:55.441025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:55.514731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881e1d7-fbc9-4157-a776-4fd2857d9df2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:55.562115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d0799-39b7-4f3a-8a86-f82ed9e22d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:55.563003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d0799-39b7-4f3a-8a86-f82ed9e22d02 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:55.563429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d0799-39b7-4f3a-8a86-f82ed9e22d02 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:55.563872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d0799-39b7-4f3a-8a86-f82ed9e22d02 None admin] PATH_INFO: `/v3/users/7840473cb1f84f61bc8ae72269e62999` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:55.565202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d0799-39b7-4f3a-8a86-f82ed9e22d02 None admin] RBAC: Authorizing `identity:get_user(user_id=7840473cb1f84f61bc8ae72269e62999)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:55.566809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d0799-39b7-4f3a-8a86-f82ed9e22d02 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:55.569425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 168/333] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:55 2022] GET /identity/v3/users/7840473cb1f84f61bc8ae72269e62999 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:55.575301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab68f4d-b917-446a-ba11-071a4d34e85d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:55.734283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab68f4d-b917-446a-ba11-071a4d34e85d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:55.752646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:55.774970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88674af-e1fb-4c1d-b7e7-d0cdc1bf366e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:55.775573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88674af-e1fb-4c1d-b7e7-d0cdc1bf366e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:55.775847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88674af-e1fb-4c1d-b7e7-d0cdc1bf366e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:55.776113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88674af-e1fb-4c1d-b7e7-d0cdc1bf366e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:55.776861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88674af-e1fb-4c1d-b7e7-d0cdc1bf366e None admin] RBAC: Adding query filter params (role.id=df2ecb74b9fe42089e380b16de5ad950, user.id=7840473cb1f84f61bc8ae72269e62999, scope.system=all) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:55.777165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88674af-e1fb-4c1d-b7e7-d0cdc1bf366e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:55.778603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88674af-e1fb-4c1d-b7e7-d0cdc1bf366e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:55.792390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 166/334] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 1 00:16:55 2022] GET /identity/v3/role_assignments?user.id=7840473cb1f84f61bc8ae72269e62999&scope.system=all&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:56.632029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c27c9-7c7c-4d2b-a77d-36f9ed75c90e None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:56.632609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c27c9-7c7c-4d2b-a77d-36f9ed75c90e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:56.632609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c27c9-7c7c-4d2b-a77d-36f9ed75c90e None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:56.634194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 169/335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:56.639041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c698f-e4ab-415f-ab52-03bcaf8088e7 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:56.639838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c698f-e4ab-415f-ab52-03bcaf8088e7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:56.640307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c698f-e4ab-415f-ab52-03bcaf8088e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:56.666753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-d06c698f-e4ab-415f-ab52-03bcaf8088e7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:56.668431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d06c698f-e4ab-415f-ab52-03bcaf8088e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:56.724734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a374b6-e482-4cb2-bab0-39bd6c9ea1a2 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:56.725061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a374b6-e482-4cb2-bab0-39bd6c9ea1a2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:56.725288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a374b6-e482-4cb2-bab0-39bd6c9ea1a2 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:56.727170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 170/336] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:16:56.732009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d78ed-8e20-4806-be8d-97d7e5cdc0ab None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:56.732291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d78ed-8e20-4806-be8d-97d7e5cdc0ab None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:56.732590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d78ed-8e20-4806-be8d-97d7e5cdc0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:56.761712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-283d78ed-8e20-4806-be8d-97d7e5cdc0ab None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:56.762925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-283d78ed-8e20-4806-be8d-97d7e5cdc0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:56.763893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 167/337] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:56.860280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 171/338] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:16:57.266074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c698f-e4ab-415f-ab52-03bcaf8088e7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:57.362436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d78ed-8e20-4806-be8d-97d7e5cdc0ab None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:57.409341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:57.415154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c698f-e4ab-415f-ab52-03bcaf8088e7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:57.439686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:57.483248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ddcaa-fd31-429a-8b32-70d19fca7509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:57.483871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ddcaa-fd31-429a-8b32-70d19fca7509 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:57.484156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ddcaa-fd31-429a-8b32-70d19fca7509 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:57.484426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ddcaa-fd31-429a-8b32-70d19fca7509 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:57.509113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ddcaa-fd31-429a-8b32-70d19fca7509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-bf7ddcaa-fd31-429a-8b32-70d19fca7509 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:57.514086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:57.514941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:57.515748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 168/339] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:16:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:57.525575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ddcaa-fd31-429a-8b32-70d19fca7509 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:57.548184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d78ed-8e20-4806-be8d-97d7e5cdc0ab None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:57.621127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb55df-62e4-458a-bd6b-e27667f51c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:57.621775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb55df-62e4-458a-bd6b-e27667f51c23 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:57.622048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb55df-62e4-458a-bd6b-e27667f51c23 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:57.622324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb55df-62e4-458a-bd6b-e27667f51c23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:57.649555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb55df-62e4-458a-bd6b-e27667f51c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-deeb55df-62e4-458a-bd6b-e27667f51c23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:57.654477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:57.655884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:57.656712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 172/340] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:57.665893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb55df-62e4-458a-bd6b-e27667f51c23 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:57.741809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ddcaa-fd31-429a-8b32-70d19fca7509 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:57.796370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695320f-460d-4e81-a2a7-45907f7bdb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:57.797096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695320f-460d-4e81-a2a7-45907f7bdb3f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:57.797375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695320f-460d-4e81-a2a7-45907f7bdb3f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:57.797644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695320f-460d-4e81-a2a7-45907f7bdb3f None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:57.798307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695320f-460d-4e81-a2a7-45907f7bdb3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:57.798633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695320f-460d-4e81-a2a7-45907f7bdb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:57.800181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695320f-460d-4e81-a2a7-45907f7bdb3f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:57.810984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 169/341] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:16:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:57.812310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21616054-87f5-451f-8241-93dcf33dbfee None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:57.812627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21616054-87f5-451f-8241-93dcf33dbfee None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:57.812903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21616054-87f5-451f-8241-93dcf33dbfee None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:57.814256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 170/342] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:57.815440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351568e-eef8-4acb-9f74-ded380ad4aad None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:57.815673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351568e-eef8-4acb-9f74-ded380ad4aad None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:57.815964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351568e-eef8-4acb-9f74-ded380ad4aad None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:57.818110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 171/343] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:57.818841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351568e-eef8-4acb-9f74-ded380ad4aad None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:57.839551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:57.842761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb55df-62e4-458a-bd6b-e27667f51c23 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:57.846632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:57.884831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee282a84-ad17-42fb-a3ec-d454cbc75bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:57.885554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee282a84-ad17-42fb-a3ec-d454cbc75bdc None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:57.885830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee282a84-ad17-42fb-a3ec-d454cbc75bdc None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:57.886205 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee282a84-ad17-42fb-a3ec-d454cbc75bdc None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:57.886881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee282a84-ad17-42fb-a3ec-d454cbc75bdc None admin] RBAC: Adding query filter params (name=member) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:57.887184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee282a84-ad17-42fb-a3ec-d454cbc75bdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:57.888686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee282a84-ad17-42fb-a3ec-d454cbc75bdc None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:57.899993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 173/344] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:57.901571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec53e5-4212-443e-aa5f-32e75f0c81a6 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:57.901937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec53e5-4212-443e-aa5f-32e75f0c81a6 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:57.902117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec53e5-4212-443e-aa5f-32e75f0c81a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:57.923612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6bec53e5-4212-443e-aa5f-32e75f0c81a6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:57.924983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6bec53e5-4212-443e-aa5f-32e75f0c81a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:57.939141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351568e-eef8-4acb-9f74-ded380ad4aad None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:57.978647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8fe97-f2f8-4a8e-a5db-bdd8dd3b0fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:57.979318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8fe97-f2f8-4a8e-a5db-bdd8dd3b0fbd None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:57.979601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8fe97-f2f8-4a8e-a5db-bdd8dd3b0fbd None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:57.979873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8fe97-f2f8-4a8e-a5db-bdd8dd3b0fbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:58.007929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 174/345] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:58.009090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e0dfb-d847-4bf3-9f6e-d2e04aba3613 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:58.009448 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e0dfb-d847-4bf3-9f6e-d2e04aba3613 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:58.009691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e0dfb-d847-4bf3-9f6e-d2e04aba3613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:58.031894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-fe8e0dfb-d847-4bf3-9f6e-d2e04aba3613 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:58.033129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fe8e0dfb-d847-4bf3-9f6e-d2e04aba3613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:58.059690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8fe97-f2f8-4a8e-a5db-bdd8dd3b0fbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:58.061240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8fe97-f2f8-4a8e-a5db-bdd8dd3b0fbd None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:58.089397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:58.158115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 175/346] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:58.159014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e0dfb-d847-4bf3-9f6e-d2e04aba3613 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:58.171128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 172/347] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:58.176681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8fe97-f2f8-4a8e-a5db-bdd8dd3b0fbd None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:58.214202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:58.287568 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8fe97-f2f8-4a8e-a5db-bdd8dd3b0fbd None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:58.327642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a58596-4bbc-4954-bd31-05d57bc722c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:58.328383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a58596-4bbc-4954-bd31-05d57bc722c3 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:58.328771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a58596-4bbc-4954-bd31-05d57bc722c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:58.329124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a58596-4bbc-4954-bd31-05d57bc722c3 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:58.330338 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a58596-4bbc-4954-bd31-05d57bc722c3 None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:58.331869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a58596-4bbc-4954-bd31-05d57bc722c3 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:58.334611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 173/348] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:58 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:58.334843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:58.340073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a58596-4bbc-4954-bd31-05d57bc722c3 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:58.344361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e0dfb-d847-4bf3-9f6e-d2e04aba3613 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:58.409274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10407163-5cf5-49c3-98e0-3beef5e2e7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:58.409896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10407163-5cf5-49c3-98e0-3beef5e2e7b0 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:58.410173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10407163-5cf5-49c3-98e0-3beef5e2e7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:58.410443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10407163-5cf5-49c3-98e0-3beef5e2e7b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:58.488245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a58596-4bbc-4954-bd31-05d57bc722c3 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:58.511717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10407163-5cf5-49c3-98e0-3beef5e2e7b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:58.513319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10407163-5cf5-49c3-98e0-3beef5e2e7b0 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:58.532489 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee41a5-9e5a-4c30-9aa5-0dcd432a663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:58.533248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee41a5-9e5a-4c30-9aa5-0dcd432a663a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:58.533601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee41a5-9e5a-4c30-9aa5-0dcd432a663a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:58.533947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee41a5-9e5a-4c30-9aa5-0dcd432a663a None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:58.541277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee41a5-9e5a-4c30-9aa5-0dcd432a663a None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:58.542795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee41a5-9e5a-4c30-9aa5-0dcd432a663a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:58.545401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 174/349] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:16:58 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:16:58.553247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee41a5-9e5a-4c30-9aa5-0dcd432a663a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:58.570617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:58.624861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:58.656220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 176/350] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:58.663165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10407163-5cf5-49c3-98e0-3beef5e2e7b0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:58.671729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee41a5-9e5a-4c30-9aa5-0dcd432a663a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:58.714879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336b2a-a7e3-49b0-b8f8-42567a4d278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:58.715836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336b2a-a7e3-49b0-b8f8-42567a4d278c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:58.716318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336b2a-a7e3-49b0-b8f8-42567a4d278c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:58.716840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336b2a-a7e3-49b0-b8f8-42567a4d278c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:58.717776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91336b2a-a7e3-49b0-b8f8-42567a4d278c None admin] RBAC: Adding query filter params (group.id=dbf5dbce88cc45ea82cb7056708a2091, role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=af2562b4c9384f15b1d6a92b56581f77) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:58.718308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91336b2a-a7e3-49b0-b8f8-42567a4d278c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:58.719963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91336b2a-a7e3-49b0-b8f8-42567a4d278c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:58.727765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 175/351] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:16:58 2022] GET /identity/v3/role_assignments?group.id=dbf5dbce88cc45ea82cb7056708a2091&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:58.733102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336b2a-a7e3-49b0-b8f8-42567a4d278c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:58.854144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10407163-5cf5-49c3-98e0-3beef5e2e7b0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:58.890645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:58.903938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336b2a-a7e3-49b0-b8f8-42567a4d278c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:58.908657 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2e909-2178-4822-b216-261717ff969d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:58.909320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2e909-2178-4822-b216-261717ff969d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:58.909601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2e909-2178-4822-b216-261717ff969d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:58.909873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2e909-2178-4822-b216-261717ff969d None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:58.911365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2e909-2178-4822-b216-261717ff969d None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:58.913101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2e909-2178-4822-b216-261717ff969d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:58.915489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 177/352] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:58 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:58.916349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2e909-2178-4822-b216-261717ff969d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:58.926664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:58.952843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fd36a-f573-4f95-8a33-a29e3fe45b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:58.954600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fd36a-f573-4f95-8a33-a29e3fe45b77 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:58.954600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fd36a-f573-4f95-8a33-a29e3fe45b77 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:58.954600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fd36a-f573-4f95-8a33-a29e3fe45b77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:58.974933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7fd36a-f573-4f95-8a33-a29e3fe45b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-fb7fd36a-f573-4f95-8a33-a29e3fe45b77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:58.981267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:58.982210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:16:58.983005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 176/353] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:58.983690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88484610-f49e-41db-89ea-8ab17efbbd40 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:58.984548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88484610-f49e-41db-89ea-8ab17efbbd40 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:58.984548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88484610-f49e-41db-89ea-8ab17efbbd40 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:58.985585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 177/354] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:58.986302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88484610-f49e-41db-89ea-8ab17efbbd40 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:59.076201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2e909-2178-4822-b216-261717ff969d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:59.121876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88484610-f49e-41db-89ea-8ab17efbbd40 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:59.127554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdabff-3f1d-4c69-b7ed-55bf38c5e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:59.128471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdabff-3f1d-4c69-b7ed-55bf38c5e464 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.128990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdabff-3f1d-4c69-b7ed-55bf38c5e464 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.129459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdabff-3f1d-4c69-b7ed-55bf38c5e464 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.146281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdabff-3f1d-4c69-b7ed-55bf38c5e464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-37cdabff-3f1d-4c69-b7ed-55bf38c5e464 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:59.151254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:59.152156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:59.153016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 178/355] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:16:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:59.154457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdabff-3f1d-4c69-b7ed-55bf38c5e464 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:59.169094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb7363-b205-46c7-8353-2acee1859e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:59.170175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7363-b205-46c7-8353-2acee1859e79 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.171090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7363-b205-46c7-8353-2acee1859e79 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.172297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb7363-b205-46c7-8353-2acee1859e79 None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.174880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb7363-b205-46c7-8353-2acee1859e79 None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:59.176352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb7363-b205-46c7-8353-2acee1859e79 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:59.178866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 178/356] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:16:58 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:59.180089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed998218-ab5a-4276-8f74-09544de60abb None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.180374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed998218-ab5a-4276-8f74-09544de60abb None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.180677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed998218-ab5a-4276-8f74-09544de60abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.199624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ed998218-ab5a-4276-8f74-09544de60abb None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:59.200866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ed998218-ab5a-4276-8f74-09544de60abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:59.228018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:59.262934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:59.294744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 179/357] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:59.295601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed998218-ab5a-4276-8f74-09544de60abb None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:59.295898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdabff-3f1d-4c69-b7ed-55bf38c5e464 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:59.344758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571de4c-045b-43de-aaeb-2bbd995a42a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:59.345745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571de4c-045b-43de-aaeb-2bbd995a42a1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.346222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571de4c-045b-43de-aaeb-2bbd995a42a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.346693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571de4c-045b-43de-aaeb-2bbd995a42a1 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.347623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571de4c-045b-43de-aaeb-2bbd995a42a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:59.349155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571de4c-045b-43de-aaeb-2bbd995a42a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:59.350813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571de4c-045b-43de-aaeb-2bbd995a42a1 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:59.362959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 179/358] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:59.364250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571de4c-045b-43de-aaeb-2bbd995a42a1 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:59.427060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed998218-ab5a-4276-8f74-09544de60abb None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:59.467327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89433d59-c8c2-4bfe-9f2b-e129edf7c715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:59.468315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89433d59-c8c2-4bfe-9f2b-e129edf7c715 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.468809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89433d59-c8c2-4bfe-9f2b-e129edf7c715 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.469277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89433d59-c8c2-4bfe-9f2b-e129edf7c715 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.470266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89433d59-c8c2-4bfe-9f2b-e129edf7c715 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:59.470832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89433d59-c8c2-4bfe-9f2b-e129edf7c715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:59.472429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89433d59-c8c2-4bfe-9f2b-e129edf7c715 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:59.481410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571de4c-045b-43de-aaeb-2bbd995a42a1 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:59.483711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 180/359] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:16:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:59.485188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d51768-5639-4915-ab2a-3d1fd4ec3d30 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.485709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d51768-5639-4915-ab2a-3d1fd4ec3d30 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.486359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d51768-5639-4915-ab2a-3d1fd4ec3d30 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.488101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 181/360] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:16:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:16:59.489111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d51768-5639-4915-ab2a-3d1fd4ec3d30 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:59.500933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:59.524923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7beccd8-fae3-4d58-868b-f88fd7e3e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:59.525519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7beccd8-fae3-4d58-868b-f88fd7e3e3df None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.525799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7beccd8-fae3-4d58-868b-f88fd7e3e3df None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.526075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7beccd8-fae3-4d58-868b-f88fd7e3e3df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.526993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7beccd8-fae3-4d58-868b-f88fd7e3e3df None admin] RBAC: Adding query filter params (group.id=dbf5dbce88cc45ea82cb7056708a2091, role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=af2562b4c9384f15b1d6a92b56581f77) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:16:59.527098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7beccd8-fae3-4d58-868b-f88fd7e3e3df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:59.528538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7beccd8-fae3-4d58-868b-f88fd7e3e3df None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:59.534503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 180/361] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:16:59 2022] GET /identity/v3/role_assignments?group.id=dbf5dbce88cc45ea82cb7056708a2091&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:59.535505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7beccd8-fae3-4d58-868b-f88fd7e3e3df None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:59.543872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:59.619376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d51768-5639-4915-ab2a-3d1fd4ec3d30 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:59.655304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7beccd8-fae3-4d58-868b-f88fd7e3e3df None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:59.684081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82459509-e3b8-4096-b1dd-6561e5f62437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:59.685339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82459509-e3b8-4096-b1dd-6561e5f62437 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.685634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82459509-e3b8-4096-b1dd-6561e5f62437 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.685908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82459509-e3b8-4096-b1dd-6561e5f62437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.703661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62164b52-7a26-4eff-a6b3-fda4afd2b397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:59.705129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62164b52-7a26-4eff-a6b3-fda4afd2b397 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.705611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62164b52-7a26-4eff-a6b3-fda4afd2b397 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.706739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62164b52-7a26-4eff-a6b3-fda4afd2b397 None admin] PATH_INFO: `/v3/users/7fb6da991055455e921ce4a3399e73b5` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.708316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62164b52-7a26-4eff-a6b3-fda4afd2b397 None admin] RBAC: Authorizing `identity:get_user(user_id=7fb6da991055455e921ce4a3399e73b5)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:59.710166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62164b52-7a26-4eff-a6b3-fda4afd2b397 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:59.712549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 181/362] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:16:59 2022] GET /identity/v3/users/7fb6da991055455e921ce4a3399e73b5 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:16:59.714062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235c204-ac8b-4424-a692-67f1ade4a38d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.714576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235c204-ac8b-4424-a692-67f1ade4a38d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.715042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235c204-ac8b-4424-a692-67f1ade4a38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.737522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f235c204-ac8b-4424-a692-67f1ade4a38d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:16:59.738794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f235c204-ac8b-4424-a692-67f1ade4a38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:16:59.779460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82459509-e3b8-4096-b1dd-6561e5f62437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:16:59.781263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82459509-e3b8-4096-b1dd-6561e5f62437 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:16:59.783570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:59.813066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 182/363] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:16:59.814001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235c204-ac8b-4424-a692-67f1ade4a38d None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:59.826963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:16:59.903701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 182/364] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:16:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:16:59.904589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82459509-e3b8-4096-b1dd-6561e5f62437 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:16:59.921746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235c204-ac8b-4424-a692-67f1ade4a38d None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:16:59.959825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c5d2-0011-4fd4-8fb0-62cdb8b9e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:16:59.960371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c5d2-0011-4fd4-8fb0-62cdb8b9e366 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:16:59.960672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c5d2-0011-4fd4-8fb0-62cdb8b9e366 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:16:59.960938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20c5d2-0011-4fd4-8fb0-62cdb8b9e366 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:16:59.977164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20c5d2-0011-4fd4-8fb0-62cdb8b9e366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-6f20c5d2-0011-4fd4-8fb0-62cdb8b9e366 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:16:59.982179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:16:59.983097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:16:59.983896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 183/365] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:16:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:16:59.984630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c5d2-0011-4fd4-8fb0-62cdb8b9e366 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:00.041732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:00.077757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82459509-e3b8-4096-b1dd-6561e5f62437 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:00.136597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8fdcf-77d4-4df3-8829-2d7756490ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:00.137260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8fdcf-77d4-4df3-8829-2d7756490ada None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:00.137544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8fdcf-77d4-4df3-8829-2d7756490ada None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:00.137946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8fdcf-77d4-4df3-8829-2d7756490ada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:00.138679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8fdcf-77d4-4df3-8829-2d7756490ada None admin] RBAC: Adding query filter params (role.id=a6588e7acf534502a191a0bca0d46c39, user.id=7fb6da991055455e921ce4a3399e73b5, scope.system=all) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:00.138986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8fdcf-77d4-4df3-8829-2d7756490ada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:00.141697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8fdcf-77d4-4df3-8829-2d7756490ada None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:00.156943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:00.164997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 183/366] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 1 00:16:59 2022] GET /identity/v3/role_assignments?user.id=7fb6da991055455e921ce4a3399e73b5&scope.system=all&role.id=a6588e7acf534502a191a0bca0d46c39 => generated 510 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:00.165959 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8fdcf-77d4-4df3-8829-2d7756490ada None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:00.168203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20c5d2-0011-4fd4-8fb0-62cdb8b9e366 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:00.238280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56921160-7079-4385-b0fb-0803c6aa9b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:00.239520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56921160-7079-4385-b0fb-0803c6aa9b1d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:00.240199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56921160-7079-4385-b0fb-0803c6aa9b1d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:00.240524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56921160-7079-4385-b0fb-0803c6aa9b1d None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:00.241788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56921160-7079-4385-b0fb-0803c6aa9b1d None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:00.243395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56921160-7079-4385-b0fb-0803c6aa9b1d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:00.247979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 184/367] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:16:59 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:00.251605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56921160-7079-4385-b0fb-0803c6aa9b1d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:00.330119 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8fdcf-77d4-4df3-8829-2d7756490ada None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:00.380188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19488cb7-9adb-43f2-aef0-0f279846c33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:00.381565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19488cb7-9adb-43f2-aef0-0f279846c33b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:00.382442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19488cb7-9adb-43f2-aef0-0f279846c33b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:00.383196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19488cb7-9adb-43f2-aef0-0f279846c33b None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:00.384294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19488cb7-9adb-43f2-aef0-0f279846c33b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:00.384850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19488cb7-9adb-43f2-aef0-0f279846c33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:00.387101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19488cb7-9adb-43f2-aef0-0f279846c33b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:00.388042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56921160-7079-4385-b0fb-0803c6aa9b1d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:00.389938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:00.399729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 184/368] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:00.401653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19488cb7-9adb-43f2-aef0-0f279846c33b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:00.443249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711ef11-aa3b-482d-89c5-cb3d8f26486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:00.443955 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711ef11-aa3b-482d-89c5-cb3d8f26486e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:00.444239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711ef11-aa3b-482d-89c5-cb3d8f26486e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:00.444528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711ef11-aa3b-482d-89c5-cb3d8f26486e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:00.465736 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711ef11-aa3b-482d-89c5-cb3d8f26486e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-5711ef11-aa3b-482d-89c5-cb3d8f26486e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:00.471468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:00.472342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 185/369] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:00.477528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711ef11-aa3b-482d-89c5-cb3d8f26486e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:00.490717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:00.549820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19488cb7-9adb-43f2-aef0-0f279846c33b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:00.598439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dbbfc-bf15-4297-b09f-a5483096efd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:00.599474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dbbfc-bf15-4297-b09f-a5483096efd7 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:00.600020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dbbfc-bf15-4297-b09f-a5483096efd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:00.600434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dbbfc-bf15-4297-b09f-a5483096efd7 None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:00.602002 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2dbbfc-bf15-4297-b09f-a5483096efd7 None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:00.602348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711ef11-aa3b-482d-89c5-cb3d8f26486e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:00.603744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2dbbfc-bf15-4297-b09f-a5483096efd7 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:00.607050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 185/370] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:00 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:00.608143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dbbfc-bf15-4297-b09f-a5483096efd7 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:00.651372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee367d5-6bd3-4d88-997b-27b2ae335a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:00.652225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee367d5-6bd3-4d88-997b-27b2ae335a0b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:00.652940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee367d5-6bd3-4d88-997b-27b2ae335a0b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:00.653378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee367d5-6bd3-4d88-997b-27b2ae335a0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:00.697553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:00.744176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dbbfc-bf15-4297-b09f-a5483096efd7 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:00.749121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee367d5-6bd3-4d88-997b-27b2ae335a0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:00.751072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee367d5-6bd3-4d88-997b-27b2ae335a0b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:00.779199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:00.794594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1fe7a-2da0-4c35-bd7c-1038ddb27d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:00.795407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1fe7a-2da0-4c35-bd7c-1038ddb27d4d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:00.795812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1fe7a-2da0-4c35-bd7c-1038ddb27d4d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:00.796190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1fe7a-2da0-4c35-bd7c-1038ddb27d4d None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:00.797093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1fe7a-2da0-4c35-bd7c-1038ddb27d4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:00.798229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1fe7a-2da0-4c35-bd7c-1038ddb27d4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:00.799097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1fe7a-2da0-4c35-bd7c-1038ddb27d4d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:00.808924 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 186/371] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:00.809907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1fe7a-2da0-4c35-bd7c-1038ddb27d4d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:00.883812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 186/372] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:00.885318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee367d5-6bd3-4d88-997b-27b2ae335a0b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:00.937166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1fe7a-2da0-4c35-bd7c-1038ddb27d4d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:00.966319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:00.984176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a043f-4d8a-4fd3-95a2-f6f52f75f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:00.984981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a043f-4d8a-4fd3-95a2-f6f52f75f02b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:00.985774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a043f-4d8a-4fd3-95a2-f6f52f75f02b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:00.985774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a043f-4d8a-4fd3-95a2-f6f52f75f02b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:00.986682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a043f-4d8a-4fd3-95a2-f6f52f75f02b None admin] RBAC: Adding query filter params (group.id=dbf5dbce88cc45ea82cb7056708a2091, role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=9a8fffb476e54c20b9662b528c1868ba) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:00.987093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a043f-4d8a-4fd3-95a2-f6f52f75f02b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:00.988802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a043f-4d8a-4fd3-95a2-f6f52f75f02b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:00.995367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 187/373] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:17:00 2022] GET /identity/v3/role_assignments?group.id=dbf5dbce88cc45ea82cb7056708a2091&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:00.996427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a043f-4d8a-4fd3-95a2-f6f52f75f02b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:01.033055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee367d5-6bd3-4d88-997b-27b2ae335a0b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:01.107572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e6a73-9a42-4d61-a7b2-daa0a26352b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:01.108659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6a73-9a42-4d61-a7b2-daa0a26352b3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.109172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6a73-9a42-4d61-a7b2-daa0a26352b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.109831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6a73-9a42-4d61-a7b2-daa0a26352b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.148446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:01.195444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e6a73-9a42-4d61-a7b2-daa0a26352b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:01.197482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e6a73-9a42-4d61-a7b2-daa0a26352b3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:01.269728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:01.287944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a043f-4d8a-4fd3-95a2-f6f52f75f02b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:01.322615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 187/374] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:01.324022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30705f9e-1b06-48be-b7e1-b51303885b79 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.324307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30705f9e-1b06-48be-b7e1-b51303885b79 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.324601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30705f9e-1b06-48be-b7e1-b51303885b79 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.328592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 188/375] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:01.331482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30705f9e-1b06-48be-b7e1-b51303885b79 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:01.347625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eccf3f-3c3d-4c47-8f7a-5731305c385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:01.348470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eccf3f-3c3d-4c47-8f7a-5731305c385f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.348911 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eccf3f-3c3d-4c47-8f7a-5731305c385f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.349181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eccf3f-3c3d-4c47-8f7a-5731305c385f None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.350612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eccf3f-3c3d-4c47-8f7a-5731305c385f None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:01.352389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eccf3f-3c3d-4c47-8f7a-5731305c385f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:01.355475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 188/376] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:00 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:01.356955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd9520-e621-433f-bd61-d3a44df53871 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.357477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd9520-e621-433f-bd61-d3a44df53871 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.357761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd9520-e621-433f-bd61-d3a44df53871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.383917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-5dcd9520-e621-433f-bd61-d3a44df53871 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:01.385169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5dcd9520-e621-433f-bd61-d3a44df53871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:01.461779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30705f9e-1b06-48be-b7e1-b51303885b79 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:01.478369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 189/377] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:01.482020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd9520-e621-433f-bd61-d3a44df53871 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:01.518715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192f053-9b8c-48f8-b799-0f0f266a7db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:01.519354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192f053-9b8c-48f8-b799-0f0f266a7db2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.519627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192f053-9b8c-48f8-b799-0f0f266a7db2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.519948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192f053-9b8c-48f8-b799-0f0f266a7db2 None admin] PATH_INFO: `/v3/projects/f6a8f68ec23146108279fe93631f855a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.523790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192f053-9b8c-48f8-b799-0f0f266a7db2 None admin] RBAC: Authorizing `identity:get_project(project_id=f6a8f68ec23146108279fe93631f855a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:01.525270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192f053-9b8c-48f8-b799-0f0f266a7db2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:01.527612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 189/378] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:01 2022] GET /identity/v3/projects/f6a8f68ec23146108279fe93631f855a => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:01.534160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192f053-9b8c-48f8-b799-0f0f266a7db2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:01.560081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:01.581116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:01.617766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd9520-e621-433f-bd61-d3a44df53871 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:01.653890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c29d93-237f-4c29-8166-dd9230c3eade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:01.654892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c29d93-237f-4c29-8166-dd9230c3eade None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.655395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c29d93-237f-4c29-8166-dd9230c3eade None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.655863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c29d93-237f-4c29-8166-dd9230c3eade None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.657215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c29d93-237f-4c29-8166-dd9230c3eade None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:01.658859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c29d93-237f-4c29-8166-dd9230c3eade None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:01.661267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 190/379] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:01 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:01.670408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c29d93-237f-4c29-8166-dd9230c3eade None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:01.701677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192f053-9b8c-48f8-b799-0f0f266a7db2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:01.768933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfc308-ce6d-40b4-91c7-827ca3f347ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:01.769703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfc308-ce6d-40b4-91c7-827ca3f347ba None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.769870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfc308-ce6d-40b4-91c7-827ca3f347ba None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.770227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfc308-ce6d-40b4-91c7-827ca3f347ba None admin] PATH_INFO: `/v3/groups/690efee7c72c4dc2a23769e57cf65253` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.781137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfc308-ce6d-40b4-91c7-827ca3f347ba None admin] RBAC: Authorizing `identity:get_group(group_id=690efee7c72c4dc2a23769e57cf65253)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:01.782576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfc308-ce6d-40b4-91c7-827ca3f347ba None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:01.784579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 190/380] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:01 2022] GET /identity/v3/groups/690efee7c72c4dc2a23769e57cf65253 => generated 229 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:01.787672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c29d93-237f-4c29-8166-dd9230c3eade None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:01.790889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfc308-ce6d-40b4-91c7-827ca3f347ba None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:01.812032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:01.826438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d31fd-be9e-4c5f-a78b-6276aef0c7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:01.827378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d31fd-be9e-4c5f-a78b-6276aef0c7cf None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.827854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d31fd-be9e-4c5f-a78b-6276aef0c7cf None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.828398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d31fd-be9e-4c5f-a78b-6276aef0c7cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.829344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d31fd-be9e-4c5f-a78b-6276aef0c7cf None admin] RBAC: Adding query filter params (group.id=dbf5dbce88cc45ea82cb7056708a2091, role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=9a8fffb476e54c20b9662b528c1868ba) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:01.829798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d31fd-be9e-4c5f-a78b-6276aef0c7cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:01.831584 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d31fd-be9e-4c5f-a78b-6276aef0c7cf None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:01.839113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 191/381] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:17:01 2022] GET /identity/v3/role_assignments?group.id=dbf5dbce88cc45ea82cb7056708a2091&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:01.924391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfce761-ce12-4f12-a9a5-92140a77071d None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.924920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfce761-ce12-4f12-a9a5-92140a77071d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.925339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfce761-ce12-4f12-a9a5-92140a77071d None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.927165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 192/382] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:17:01.932980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:01.944630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfc308-ce6d-40b4-91c7-827ca3f347ba None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:01.946260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05997-ec51-494d-999a-12e92d839226 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.947178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05997-ec51-494d-999a-12e92d839226 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.947652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05997-ec51-494d-999a-12e92d839226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.970407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-acd05997-ec51-494d-999a-12e92d839226 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:01.971982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-acd05997-ec51-494d-999a-12e92d839226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:01.985594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81835e81-c37a-4acc-9ca2-36e679d5186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:01.986211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81835e81-c37a-4acc-9ca2-36e679d5186a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:01.986486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81835e81-c37a-4acc-9ca2-36e679d5186a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:01.986757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81835e81-c37a-4acc-9ca2-36e679d5186a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:01.987494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81835e81-c37a-4acc-9ca2-36e679d5186a None admin] RBAC: Adding query filter params (group.id=690efee7c72c4dc2a23769e57cf65253, role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=f6a8f68ec23146108279fe93631f855a) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:01.987798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81835e81-c37a-4acc-9ca2-36e679d5186a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:01.990074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81835e81-c37a-4acc-9ca2-36e679d5186a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:01.996368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 191/383] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:17:01 2022] GET /identity/v3/role_assignments?group.id=690efee7c72c4dc2a23769e57cf65253&scope.project.id=f6a8f68ec23146108279fe93631f855a&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:01.997186 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81835e81-c37a-4acc-9ca2-36e679d5186a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:02.067740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 193/384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:02.116947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81835e81-c37a-4acc-9ca2-36e679d5186a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:02.153580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e242a46-7809-4a71-914c-1bebbdf47606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:02.154224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242a46-7809-4a71-914c-1bebbdf47606 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:02.154498 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242a46-7809-4a71-914c-1bebbdf47606 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:02.154766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e242a46-7809-4a71-914c-1bebbdf47606 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:02.166404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e242a46-7809-4a71-914c-1bebbdf47606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-2e242a46-7809-4a71-914c-1bebbdf47606 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:02.171661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:02.172568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:02.173352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 192/385] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:02.179444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05997-ec51-494d-999a-12e92d839226 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:02.348077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05997-ec51-494d-999a-12e92d839226 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:02.368662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:02.397222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e05645-743a-4a98-9e14-c50415c40e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:02.397840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e05645-743a-4a98-9e14-c50415c40e0b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:02.398117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e05645-743a-4a98-9e14-c50415c40e0b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:02.398389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e05645-743a-4a98-9e14-c50415c40e0b None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:02.399068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e05645-743a-4a98-9e14-c50415c40e0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:02.399367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e05645-743a-4a98-9e14-c50415c40e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:02.400776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e05645-743a-4a98-9e14-c50415c40e0b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:02.409854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 194/386] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:02.415995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e242a46-7809-4a71-914c-1bebbdf47606 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:02.452012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:02.525317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e05645-743a-4a98-9e14-c50415c40e0b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:02.559832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e242a46-7809-4a71-914c-1bebbdf47606 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:02.647318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdd1b5-9317-40f0-8554-78c8394d43b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:02.647880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdd1b5-9317-40f0-8554-78c8394d43b8 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:02.648162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdd1b5-9317-40f0-8554-78c8394d43b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:02.648437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acdd1b5-9317-40f0-8554-78c8394d43b8 None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:02.651963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdd1b5-9317-40f0-8554-78c8394d43b8 None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:02.653721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acdd1b5-9317-40f0-8554-78c8394d43b8 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:02.655489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 193/387] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:02 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:02.657158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e05645-743a-4a98-9e14-c50415c40e0b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:02.661689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdd1b5-9317-40f0-8554-78c8394d43b8 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:02.696113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afece91-0914-4893-a965-58a165bfbefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:02.697317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afece91-0914-4893-a965-58a165bfbefb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:02.698164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afece91-0914-4893-a965-58a165bfbefb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:02.698635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afece91-0914-4893-a965-58a165bfbefb None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:02.711656 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afece91-0914-4893-a965-58a165bfbefb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-9afece91-0914-4893-a965-58a165bfbefb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:02.718235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:02.719116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:17:02.719912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:02.722601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 195/388] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:02.729904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afece91-0914-4893-a965-58a165bfbefb None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:02.803908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:02.834569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:02.847148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acdd1b5-9317-40f0-8554-78c8394d43b8 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:02.894908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa635dd-1de1-4625-a6d5-1757f0cecb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:02.896893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa635dd-1de1-4625-a6d5-1757f0cecb98 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:02.898495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa635dd-1de1-4625-a6d5-1757f0cecb98 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:02.900980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa635dd-1de1-4625-a6d5-1757f0cecb98 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:02.903078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa635dd-1de1-4625-a6d5-1757f0cecb98 None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:02.904741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa635dd-1de1-4625-a6d5-1757f0cecb98 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:02.908266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 194/389] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:02 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:02.917588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa635dd-1de1-4625-a6d5-1757f0cecb98 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:02.994610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afece91-0914-4893-a965-58a165bfbefb None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:03.057413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa635dd-1de1-4625-a6d5-1757f0cecb98 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:03.095848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c15dd1-a880-4182-9b06-9b8248069f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:03.096913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c15dd1-a880-4182-9b06-9b8248069f4f None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:03.097446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c15dd1-a880-4182-9b06-9b8248069f4f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:03.097916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c15dd1-a880-4182-9b06-9b8248069f4f None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77/groups/dbf5dbce88cc45ea82cb7056708a2091/roles/2986c98e353a489592c69ada5c3c2aaf` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:03.101440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c15dd1-a880-4182-9b06-9b8248069f4f None admin] RBAC: Authorizing `identity:create_grant(project_id=af2562b4c9384f15b1d6a92b56581f77, group_id=dbf5dbce88cc45ea82cb7056708a2091, role_id=2986c98e353a489592c69ada5c3c2aaf)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:03.103162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c15dd1-a880-4182-9b06-9b8248069f4f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:03.103957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4452c3-cf0b-4eb2-8288-1e3ac2724058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:03.104740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4452c3-cf0b-4eb2-8288-1e3ac2724058 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:03.105121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4452c3-cf0b-4eb2-8288-1e3ac2724058 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:03.105495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4452c3-cf0b-4eb2-8288-1e3ac2724058 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:03.106271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4452c3-cf0b-4eb2-8288-1e3ac2724058 None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:03.106675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4452c3-cf0b-4eb2-8288-1e3ac2724058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:03.108271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4452c3-cf0b-4eb2-8288-1e3ac2724058 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:03.124380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 195/390] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 00:17:02 2022] PUT /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77/groups/dbf5dbce88cc45ea82cb7056708a2091/roles/2986c98e353a489592c69ada5c3c2aaf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:03.136034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 196/391] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:17:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:03.141479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c15dd1-a880-4182-9b06-9b8248069f4f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:03.168988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:03.276566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c15dd1-a880-4182-9b06-9b8248069f4f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:03.332284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bef00-2ac6-401c-a3c3-18e47f8ce751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:03.333165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bef00-2ac6-401c-a3c3-18e47f8ce751 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:03.333446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bef00-2ac6-401c-a3c3-18e47f8ce751 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:03.333717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bef00-2ac6-401c-a3c3-18e47f8ce751 None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:03.337381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62bef00-2ac6-401c-a3c3-18e47f8ce751 None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:03.338947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62bef00-2ac6-401c-a3c3-18e47f8ce751 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:03.341073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 196/392] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:03 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:03.347494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4452c3-cf0b-4eb2-8288-1e3ac2724058 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:03.493121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:03.502493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4452c3-cf0b-4eb2-8288-1e3ac2724058 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:03.544029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6bde2-d4d2-49c8-bcb3-5f498106caaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:03.544776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6bde2-d4d2-49c8-bcb3-5f498106caaa None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:03.545055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6bde2-d4d2-49c8-bcb3-5f498106caaa None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:03.545326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6bde2-d4d2-49c8-bcb3-5f498106caaa None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:03.546472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6bde2-d4d2-49c8-bcb3-5f498106caaa None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:03.547927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6bde2-d4d2-49c8-bcb3-5f498106caaa None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:03.551702 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 197/393] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:03 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:03.557397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bef00-2ac6-401c-a3c3-18e47f8ce751 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:03.586174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600629a5-72fc-4b9c-9385-f8606289dd77 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:03.586475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600629a5-72fc-4b9c-9385-f8606289dd77 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:03.586716 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600629a5-72fc-4b9c-9385-f8606289dd77 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:03.588216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 198/394] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:17:03.593065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2f408-85e4-43e2-8026-389ad9a12dd4 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:03.593345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2f408-85e4-43e2-8026-389ad9a12dd4 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:03.593622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2f408-85e4-43e2-8026-389ad9a12dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:03.622598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e2b2f408-85e4-43e2-8026-389ad9a12dd4 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:03.626175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e2b2f408-85e4-43e2-8026-389ad9a12dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:03.692866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:03.710168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bef00-2ac6-401c-a3c3-18e47f8ce751 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:03.747510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 199/395] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:03.747909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a532e-fb9f-44d9-a4fd-f45de6a2b4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:03.748677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a532e-fb9f-44d9-a4fd-f45de6a2b4aa None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:03.749169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a532e-fb9f-44d9-a4fd-f45de6a2b4aa None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:03.749450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a532e-fb9f-44d9-a4fd-f45de6a2b4aa None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77/groups/dbf5dbce88cc45ea82cb7056708a2091/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:03.751453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a532e-fb9f-44d9-a4fd-f45de6a2b4aa None admin] RBAC: Authorizing `identity:create_grant(project_id=af2562b4c9384f15b1d6a92b56581f77, group_id=dbf5dbce88cc45ea82cb7056708a2091, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:03.753266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a532e-fb9f-44d9-a4fd-f45de6a2b4aa None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:03.769658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 197/396] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 00:17:03 2022] PUT /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77/groups/dbf5dbce88cc45ea82cb7056708a2091/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:04.141012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2f408-85e4-43e2-8026-389ad9a12dd4 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:04.270994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:04.278180 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2f408-85e4-43e2-8026-389ad9a12dd4 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:04.288972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3915a-70ff-4ecb-bba7-f4290ad3db3f None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:04.290279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3915a-70ff-4ecb-bba7-f4290ad3db3f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:04.290762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3915a-70ff-4ecb-bba7-f4290ad3db3f None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:04.294634 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 198/397] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:04.299907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c862e-14ac-49bf-99b4-8c1f8234f978 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:04.300431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c862e-14ac-49bf-99b4-8c1f8234f978 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:04.300939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c862e-14ac-49bf-99b4-8c1f8234f978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:04.320512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898856b-14e6-40fa-ba64-6440467b2da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:04.321167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898856b-14e6-40fa-ba64-6440467b2da1 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:04.321450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898856b-14e6-40fa-ba64-6440467b2da1 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:04.321725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898856b-14e6-40fa-ba64-6440467b2da1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:04.333330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898856b-14e6-40fa-ba64-6440467b2da1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:04.334664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-361c862e-14ac-49bf-99b4-8c1f8234f978 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:04.335883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-361c862e-14ac-49bf-99b4-8c1f8234f978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-3898856b-14e6-40fa-ba64-6440467b2da1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:04.338452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:04.339374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:04.340202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 200/398] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:04.362600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898856b-14e6-40fa-ba64-6440467b2da1 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:04.435396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 199/399] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:04.451307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223b9e7-7acb-4245-8909-0b1cabd56d5a None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:04.451626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223b9e7-7acb-4245-8909-0b1cabd56d5a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:04.451854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223b9e7-7acb-4245-8909-0b1cabd56d5a None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:04.453656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 200/400] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:17:04.459284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60be49-395f-4f34-afe2-6d5f68aa1717 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:04.459554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60be49-395f-4f34-afe2-6d5f68aa1717 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:04.459884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60be49-395f-4f34-afe2-6d5f68aa1717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:04.480706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-dc60be49-395f-4f34-afe2-6d5f68aa1717 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:04.482252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-dc60be49-395f-4f34-afe2-6d5f68aa1717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:04.534483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898856b-14e6-40fa-ba64-6440467b2da1 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:04.559643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 201/401] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:04.606576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838130c5-6ffd-4ebe-8782-46a597aa1be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:04.607188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838130c5-6ffd-4ebe-8782-46a597aa1be0 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:04.607464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838130c5-6ffd-4ebe-8782-46a597aa1be0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:04.607735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838130c5-6ffd-4ebe-8782-46a597aa1be0 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:04.608410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838130c5-6ffd-4ebe-8782-46a597aa1be0 None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:04.608736 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838130c5-6ffd-4ebe-8782-46a597aa1be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:04.610204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838130c5-6ffd-4ebe-8782-46a597aa1be0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:04.622774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 201/402] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:17:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:04.628461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60be49-395f-4f34-afe2-6d5f68aa1717 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:04.633477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:04.736694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838130c5-6ffd-4ebe-8782-46a597aa1be0 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:04.768304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60be49-395f-4f34-afe2-6d5f68aa1717 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:04.772928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:04.810691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7939d6-15ab-42cb-8576-33945930e0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:04.811462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7939d6-15ab-42cb-8576-33945930e0fe None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:04.811746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7939d6-15ab-42cb-8576-33945930e0fe None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:04.812018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7939d6-15ab-42cb-8576-33945930e0fe None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:04.813158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7939d6-15ab-42cb-8576-33945930e0fe None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:04.814767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7939d6-15ab-42cb-8576-33945930e0fe None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:04.816412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 202/403] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:04 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:04.823302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7939d6-15ab-42cb-8576-33945930e0fe None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:04.913833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:04.925669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838130c5-6ffd-4ebe-8782-46a597aa1be0 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:04.954911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7939d6-15ab-42cb-8576-33945930e0fe None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:04.968337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91561c15-ae9f-41dc-8742-b89e8e769481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:04.969454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91561c15-ae9f-41dc-8742-b89e8e769481 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:04.970052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91561c15-ae9f-41dc-8742-b89e8e769481 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:04.970506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91561c15-ae9f-41dc-8742-b89e8e769481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:04.983459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91561c15-ae9f-41dc-8742-b89e8e769481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-91561c15-ae9f-41dc-8742-b89e8e769481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:04.991309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:04.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:04.993060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:04.998649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 202/404] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:05.002140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03ebd0-ede4-449e-8a37-9ccaa65273b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:05.002793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03ebd0-ede4-449e-8a37-9ccaa65273b4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.003072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03ebd0-ede4-449e-8a37-9ccaa65273b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.003348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03ebd0-ede4-449e-8a37-9ccaa65273b4 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.003571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91561c15-ae9f-41dc-8742-b89e8e769481 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:05.005170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03ebd0-ede4-449e-8a37-9ccaa65273b4 None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:05.006636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03ebd0-ede4-449e-8a37-9ccaa65273b4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:05.009928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 203/405] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:04 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:05.015960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03ebd0-ede4-449e-8a37-9ccaa65273b4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:05.101149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:05.125544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91561c15-ae9f-41dc-8742-b89e8e769481 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:05.142509 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03ebd0-ede4-449e-8a37-9ccaa65273b4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:05.166210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea21ec5-66dd-4ad4-8e23-8c3e459043f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:05.167690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea21ec5-66dd-4ad4-8e23-8c3e459043f0 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.168414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea21ec5-66dd-4ad4-8e23-8c3e459043f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.168917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea21ec5-66dd-4ad4-8e23-8c3e459043f0 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.169877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea21ec5-66dd-4ad4-8e23-8c3e459043f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:05.170594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea21ec5-66dd-4ad4-8e23-8c3e459043f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:05.172236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea21ec5-66dd-4ad4-8e23-8c3e459043f0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:05.179565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22770-93c8-43d1-9954-25e627d84737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:05.180019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22770-93c8-43d1-9954-25e627d84737 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.180301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22770-93c8-43d1-9954-25e627d84737 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.180601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22770-93c8-43d1-9954-25e627d84737 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba/groups/dbf5dbce88cc45ea82cb7056708a2091/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.182775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22770-93c8-43d1-9954-25e627d84737 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a8fffb476e54c20b9662b528c1868ba, group_id=dbf5dbce88cc45ea82cb7056708a2091, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:05.184231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22770-93c8-43d1-9954-25e627d84737 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:05.184483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 203/406] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:05.185533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea21ec5-66dd-4ad4-8e23-8c3e459043f0 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:05.205734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:05.215491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 204/407] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 00:17:05 2022] PUT /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba/groups/dbf5dbce88cc45ea82cb7056708a2091/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:17:05.217108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22770-93c8-43d1-9954-25e627d84737 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:05.353896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22770-93c8-43d1-9954-25e627d84737 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:05.409560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea21ec5-66dd-4ad4-8e23-8c3e459043f0 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:05.412725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95657cb5-3243-4cff-90a9-e4fadbf30ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:05.413673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95657cb5-3243-4cff-90a9-e4fadbf30ce3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.414165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95657cb5-3243-4cff-90a9-e4fadbf30ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.414636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95657cb5-3243-4cff-90a9-e4fadbf30ce3 None admin] PATH_INFO: `/v3/groups/690efee7c72c4dc2a23769e57cf65253` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.417035 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95657cb5-3243-4cff-90a9-e4fadbf30ce3 None admin] RBAC: Authorizing `identity:get_group(group_id=690efee7c72c4dc2a23769e57cf65253)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:05.419876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95657cb5-3243-4cff-90a9-e4fadbf30ce3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:05.426623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 205/408] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:05 2022] GET /identity/v3/groups/690efee7c72c4dc2a23769e57cf65253 => generated 229 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:05.432547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95657cb5-3243-4cff-90a9-e4fadbf30ce3 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:05.463951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d3a99-84d3-4234-952a-8d9d1426ddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:05.464819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d3a99-84d3-4234-952a-8d9d1426ddbb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.465097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d3a99-84d3-4234-952a-8d9d1426ddbb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.465373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d3a99-84d3-4234-952a-8d9d1426ddbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.477812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:05.485035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d3a99-84d3-4234-952a-8d9d1426ddbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-834d3a99-84d3-4234-952a-8d9d1426ddbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:05.491146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:05.492055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:05.492896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 204/409] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:05.501132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d3a99-84d3-4234-952a-8d9d1426ddbb None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:05.546453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:05.608679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95657cb5-3243-4cff-90a9-e4fadbf30ce3 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:05.679162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e864f1e-3dd7-4cf3-bf6d-a87341da60ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:05.679810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e864f1e-3dd7-4cf3-bf6d-a87341da60ba None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.680086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e864f1e-3dd7-4cf3-bf6d-a87341da60ba None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.680359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e864f1e-3dd7-4cf3-bf6d-a87341da60ba None admin] PATH_INFO: `/v3/projects/f6a8f68ec23146108279fe93631f855a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.682143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e864f1e-3dd7-4cf3-bf6d-a87341da60ba None admin] RBAC: Authorizing `identity:get_project(project_id=f6a8f68ec23146108279fe93631f855a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:05.683596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e864f1e-3dd7-4cf3-bf6d-a87341da60ba None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:05.687915 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 206/410] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:05 2022] GET /identity/v3/projects/f6a8f68ec23146108279fe93631f855a => generated 344 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:05.691456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e864f1e-3dd7-4cf3-bf6d-a87341da60ba None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:05.697060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d3a99-84d3-4234-952a-8d9d1426ddbb None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:05.769068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26650295-1fdb-4a86-ad7e-f3dde317f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:05.769688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650295-1fdb-4a86-ad7e-f3dde317f4ae None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.770558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650295-1fdb-4a86-ad7e-f3dde317f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.770829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26650295-1fdb-4a86-ad7e-f3dde317f4ae None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.771507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26650295-1fdb-4a86-ad7e-f3dde317f4ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:05.771806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26650295-1fdb-4a86-ad7e-f3dde317f4ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:05.773243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26650295-1fdb-4a86-ad7e-f3dde317f4ae None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:05.784538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 205/411] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:05.790620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26650295-1fdb-4a86-ad7e-f3dde317f4ae None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:05.865850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e864f1e-3dd7-4cf3-bf6d-a87341da60ba None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:05.907808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ddf37-940d-4857-a6aa-6f91d19f44e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:05.908889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ddf37-940d-4857-a6aa-6f91d19f44e5 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.909340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ddf37-940d-4857-a6aa-6f91d19f44e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.910214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ddf37-940d-4857-a6aa-6f91d19f44e5 None admin] PATH_INFO: `/v3/projects/f6a8f68ec23146108279fe93631f855a/groups/690efee7c72c4dc2a23769e57cf65253/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.912844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ddf37-940d-4857-a6aa-6f91d19f44e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6a8f68ec23146108279fe93631f855a, group_id=690efee7c72c4dc2a23769e57cf65253, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:05.914721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ddf37-940d-4857-a6aa-6f91d19f44e5 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:05.917193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:05.935011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 207/412] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 00:17:05 2022] PUT /identity/v3/projects/f6a8f68ec23146108279fe93631f855a/groups/690efee7c72c4dc2a23769e57cf65253/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:05.942407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:05.946111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131942c4-f48b-4398-8fe0-a2885291f8d1 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.946978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131942c4-f48b-4398-8fe0-a2885291f8d1 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.947449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131942c4-f48b-4398-8fe0-a2885291f8d1 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.950241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 208/413] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:05.957995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f844a2-4918-4aab-8f5f-033bb3eaff2b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:05.958760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f844a2-4918-4aab-8f5f-033bb3eaff2b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:05.959239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f844a2-4918-4aab-8f5f-033bb3eaff2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:05.983477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-34f844a2-4918-4aab-8f5f-033bb3eaff2b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:05.985097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-34f844a2-4918-4aab-8f5f-033bb3eaff2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:06.050812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26650295-1fdb-4a86-ad7e-f3dde317f4ae None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:06.077064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 209/414] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:06.132206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3115c01-ddba-4e50-a480-6396048c2aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:06.132833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3115c01-ddba-4e50-a480-6396048c2aea None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:06.133082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3115c01-ddba-4e50-a480-6396048c2aea None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:06.133356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3115c01-ddba-4e50-a480-6396048c2aea None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:06.134487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3115c01-ddba-4e50-a480-6396048c2aea None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:06.135924 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3115c01-ddba-4e50-a480-6396048c2aea None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:06.137832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 206/415] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:05 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:06.143351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f844a2-4918-4aab-8f5f-033bb3eaff2b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:06.164740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa11e-1583-439a-bffd-1f02e2ad0837 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:06.165020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa11e-1583-439a-bffd-1f02e2ad0837 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:06.165306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa11e-1583-439a-bffd-1f02e2ad0837 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:06.166896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 207/416] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:17:06.171641 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a82e6-f895-4d97-b4a5-4f6c579fe287 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:06.171927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a82e6-f895-4d97-b4a5-4f6c579fe287 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:06.172246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a82e6-f895-4d97-b4a5-4f6c579fe287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:06.197363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-eb4a82e6-f895-4d97-b4a5-4f6c579fe287 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:06.198555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-eb4a82e6-f895-4d97-b4a5-4f6c579fe287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:06.289947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f844a2-4918-4aab-8f5f-033bb3eaff2b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:06.305013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 208/417] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:06.348489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:06.365922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbace0-b10c-4c5e-8382-185dc76af327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:06.366639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbace0-b10c-4c5e-8382-185dc76af327 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:06.366835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbace0-b10c-4c5e-8382-185dc76af327 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:06.367104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbace0-b10c-4c5e-8382-185dc76af327 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:06.373181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbace0-b10c-4c5e-8382-185dc76af327 None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:06.374824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbace0-b10c-4c5e-8382-185dc76af327 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:06.376949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 210/418] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:06 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:06.383696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a82e6-f895-4d97-b4a5-4f6c579fe287 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:06.464763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbace0-b10c-4c5e-8382-185dc76af327 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:06.498029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:06.576337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4a82e6-f895-4d97-b4a5-4f6c579fe287 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:06.630286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbace0-b10c-4c5e-8382-185dc76af327 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:06.639883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ae1cb-150f-49d2-9ac3-a2f6d792dee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:06.640779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ae1cb-150f-49d2-9ac3-a2f6d792dee7 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:06.641250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ae1cb-150f-49d2-9ac3-a2f6d792dee7 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:06.641758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ae1cb-150f-49d2-9ac3-a2f6d792dee7 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba/groups/dbf5dbce88cc45ea82cb7056708a2091/roles/2986c98e353a489592c69ada5c3c2aaf` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:06.649727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ae1cb-150f-49d2-9ac3-a2f6d792dee7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a8fffb476e54c20b9662b528c1868ba, group_id=dbf5dbce88cc45ea82cb7056708a2091, role_id=2986c98e353a489592c69ada5c3c2aaf)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:06.651460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ae1cb-150f-49d2-9ac3-a2f6d792dee7 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:06.668556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 209/419] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 1 00:17:06 2022] PUT /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba/groups/dbf5dbce88cc45ea82cb7056708a2091/roles/2986c98e353a489592c69ada5c3c2aaf => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:06.669760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ae1cb-150f-49d2-9ac3-a2f6d792dee7 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:06.669876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b72c6-a9d4-4c51-9652-49c31e3450b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:06.670892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b72c6-a9d4-4c51-9652-49c31e3450b0 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:06.671420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b72c6-a9d4-4c51-9652-49c31e3450b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:06.671785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b72c6-a9d4-4c51-9652-49c31e3450b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:06.686595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b72c6-a9d4-4c51-9652-49c31e3450b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-651b72c6-a9d4-4c51-9652-49c31e3450b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:06.692911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:06.694001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:06.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:06.704383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 211/420] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:06.704383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b72c6-a9d4-4c51-9652-49c31e3450b0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:06.825916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:06.867179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ae1cb-150f-49d2-9ac3-a2f6d792dee7 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:06.892733 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:06.897208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b72c6-a9d4-4c51-9652-49c31e3450b0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:06.930124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ed7ad-bb49-4251-8837-bacf615d2933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:06.931037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ed7ad-bb49-4251-8837-bacf615d2933 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:06.931456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ed7ad-bb49-4251-8837-bacf615d2933 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:06.931907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ed7ad-bb49-4251-8837-bacf615d2933 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:06.934031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1d7b8-b2fd-46a9-8e75-5052fca9119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:06.934806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1d7b8-b2fd-46a9-8e75-5052fca9119b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:06.935153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1d7b8-b2fd-46a9-8e75-5052fca9119b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:06.935502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1d7b8-b2fd-46a9-8e75-5052fca9119b None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:06.936257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1d7b8-b2fd-46a9-8e75-5052fca9119b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:06.936666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1d7b8-b2fd-46a9-8e75-5052fca9119b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:06.938112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1d7b8-b2fd-46a9-8e75-5052fca9119b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:06.948384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 212/421] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:06.949412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ed7ad-bb49-4251-8837-bacf615d2933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-f96ed7ad-bb49-4251-8837-bacf615d2933 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:06.959771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:06.960705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:17:06.961475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:06.963762 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 210/422] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:06.965503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1d7b8-b2fd-46a9-8e75-5052fca9119b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:06.975715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ed7ad-bb49-4251-8837-bacf615d2933 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:07.080439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1d7b8-b2fd-46a9-8e75-5052fca9119b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:07.095734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:07.120362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ed7ad-bb49-4251-8837-bacf615d2933 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:07.149322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f483c-3db5-4a7d-8b19-7843c386f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:07.149972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f483c-3db5-4a7d-8b19-7843c386f0ca None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:07.150267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f483c-3db5-4a7d-8b19-7843c386f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:07.150533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f483c-3db5-4a7d-8b19-7843c386f0ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:07.159821 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd8177-6ed8-4926-af6b-854dfb4d1a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:07.161428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd8177-6ed8-4926-af6b-854dfb4d1a01 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:07.161917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd8177-6ed8-4926-af6b-854dfb4d1a01 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:07.162398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd8177-6ed8-4926-af6b-854dfb4d1a01 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:07.163259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd8177-6ed8-4926-af6b-854dfb4d1a01 None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:07.163752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd8177-6ed8-4926-af6b-854dfb4d1a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:07.165425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd8177-6ed8-4926-af6b-854dfb4d1a01 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:07.175130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 211/423] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:17:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:07.184831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd8177-6ed8-4926-af6b-854dfb4d1a01 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:07.233314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:07.306967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd8177-6ed8-4926-af6b-854dfb4d1a01 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:07.347366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb6413-b3fb-46ef-ae50-d8f9fa9fe285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:07.348416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb6413-b3fb-46ef-ae50-d8f9fa9fe285 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:07.348873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb6413-b3fb-46ef-ae50-d8f9fa9fe285 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:07.349295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb6413-b3fb-46ef-ae50-d8f9fa9fe285 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:07.361793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f483c-3db5-4a7d-8b19-7843c386f0ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:07.363280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f483c-3db5-4a7d-8b19-7843c386f0ca None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:07.430615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb6413-b3fb-46ef-ae50-d8f9fa9fe285 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:07.432363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb6413-b3fb-46ef-ae50-d8f9fa9fe285 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:07.479992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:07.549223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 212/424] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:07.555815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb6413-b3fb-46ef-ae50-d8f9fa9fe285 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:07.609323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 213/425] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:07.610612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824b668-d78f-4be8-ae1e-9ec4d37470fb None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:07.610899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824b668-d78f-4be8-ae1e-9ec4d37470fb None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:07.611175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824b668-d78f-4be8-ae1e-9ec4d37470fb None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:07.616326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 214/426] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:07.617630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795f0b-4c06-4166-a3b4-a48feb97e428 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:07.618261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795f0b-4c06-4166-a3b4-a48feb97e428 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:07.618261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795f0b-4c06-4166-a3b4-a48feb97e428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:07.640677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-32795f0b-4c06-4166-a3b4-a48feb97e428 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:07.642374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-32795f0b-4c06-4166-a3b4-a48feb97e428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:07.655959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:07.686193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb6413-b3fb-46ef-ae50-d8f9fa9fe285 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:07.737166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa340ab9-f240-418b-a6b0-b6c4034a781b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:07.737798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa340ab9-f240-418b-a6b0-b6c4034a781b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:07.738070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa340ab9-f240-418b-a6b0-b6c4034a781b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:07.738343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa340ab9-f240-418b-a6b0-b6c4034a781b None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:07.740165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa340ab9-f240-418b-a6b0-b6c4034a781b None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:07.741724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa340ab9-f240-418b-a6b0-b6c4034a781b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:07.744751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 215/428] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:07.745673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32795f0b-4c06-4166-a3b4-a48feb97e428 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:07.747816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 213/428] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:07 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:07.749325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa340ab9-f240-418b-a6b0-b6c4034a781b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:07.767502 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:07.925434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa340ab9-f240-418b-a6b0-b6c4034a781b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:07.933849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32795f0b-4c06-4166-a3b4-a48feb97e428 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:07.977288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc98b4-355b-46d7-815e-cec6f870ffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:07.977831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc98b4-355b-46d7-815e-cec6f870ffb8 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:07.978104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc98b4-355b-46d7-815e-cec6f870ffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:07.978371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc98b4-355b-46d7-815e-cec6f870ffb8 None admin] PATH_INFO: `/v3/projects/af2562b4c9384f15b1d6a92b56581f77` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:07.979486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc98b4-355b-46d7-815e-cec6f870ffb8 None admin] RBAC: Authorizing `identity:get_project(project_id=af2562b4c9384f15b1d6a92b56581f77)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:07.980988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc98b4-355b-46d7-815e-cec6f870ffb8 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:07.983033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 216/429] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:07 2022] GET /identity/v3/projects/af2562b4c9384f15b1d6a92b56581f77 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:07.989429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc98b4-355b-46d7-815e-cec6f870ffb8 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:08.010239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:08.011663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6680b-67c0-45e5-82bd-f78c7317091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:08.012298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6680b-67c0-45e5-82bd-f78c7317091d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.012599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6680b-67c0-45e5-82bd-f78c7317091d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.012905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6680b-67c0-45e5-82bd-f78c7317091d None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.014894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6680b-67c0-45e5-82bd-f78c7317091d None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:08.016350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6680b-67c0-45e5-82bd-f78c7317091d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:08.019087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 214/430] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:07 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:08.025412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6680b-67c0-45e5-82bd-f78c7317091d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:08.128172 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:08.139044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc98b4-355b-46d7-815e-cec6f870ffb8 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:08.153582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6680b-67c0-45e5-82bd-f78c7317091d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:08.189280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758badeb-f012-4df4-a719-010d19e8c0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:08.189956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758badeb-f012-4df4-a719-010d19e8c0ec None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.190196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758badeb-f012-4df4-a719-010d19e8c0ec None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.190404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758badeb-f012-4df4-a719-010d19e8c0ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.191147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758badeb-f012-4df4-a719-010d19e8c0ec None admin] RBAC: Adding query filter params (group.id=dbf5dbce88cc45ea82cb7056708a2091, role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=af2562b4c9384f15b1d6a92b56581f77) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:08.191435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758badeb-f012-4df4-a719-010d19e8c0ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:08.193613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758badeb-f012-4df4-a719-010d19e8c0ec None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:08.193925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eff55b-7364-4e6c-9501-deff927fd610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:08.194782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eff55b-7364-4e6c-9501-deff927fd610 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.195689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eff55b-7364-4e6c-9501-deff927fd610 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.196164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eff55b-7364-4e6c-9501-deff927fd610 None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.198362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eff55b-7364-4e6c-9501-deff927fd610 None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:08.199769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 215/431] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:17:08 2022] GET /identity/v3/role_assignments?group.id=dbf5dbce88cc45ea82cb7056708a2091&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:08.200245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eff55b-7364-4e6c-9501-deff927fd610 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:08.200595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758badeb-f012-4df4-a719-010d19e8c0ec None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:08.202384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 217/432] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:07 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:08.209878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eff55b-7364-4e6c-9501-deff927fd610 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:08.361296 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:08.389321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758badeb-f012-4df4-a719-010d19e8c0ec None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:08.395811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eff55b-7364-4e6c-9501-deff927fd610 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:08.434491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0da41-a767-45d6-b6a7-9738a895f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:08.435132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0da41-a767-45d6-b6a7-9738a895f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.435411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0da41-a767-45d6-b6a7-9738a895f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.435679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0da41-a767-45d6-b6a7-9738a895f0c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.437440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2d341-a8a4-40ad-a58c-85241be51f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:08.437578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2d341-a8a4-40ad-a58c-85241be51f5f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.437763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2d341-a8a4-40ad-a58c-85241be51f5f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.438011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2d341-a8a4-40ad-a58c-85241be51f5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.438741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2d341-a8a4-40ad-a58c-85241be51f5f None admin] RBAC: Adding query filter params (group.id=dbf5dbce88cc45ea82cb7056708a2091, role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=af2562b4c9384f15b1d6a92b56581f77) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:08.439045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2d341-a8a4-40ad-a58c-85241be51f5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:08.440485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2d341-a8a4-40ad-a58c-85241be51f5f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:08.447729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:08.449422 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0da41-a767-45d6-b6a7-9738a895f0c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:08.456535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 218/433] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:17:08 2022] GET /identity/v3/role_assignments?group.id=dbf5dbce88cc45ea82cb7056708a2091&scope.project.id=af2562b4c9384f15b1d6a92b56581f77&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 611 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-d0f0da41-a767-45d6-b6a7-9738a895f0c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:08.456694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:08.457597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 00:17:08.458588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:08.459266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836f5cf-f952-4e95-aad1-625d23b918ae None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.459266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836f5cf-f952-4e95-aad1-625d23b918ae None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.459266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836f5cf-f952-4e95-aad1-625d23b918ae None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.459433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 216/434] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:08.459495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 219/435] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:08.462594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0da41-a767-45d6-b6a7-9738a895f0c3 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:08.462959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cadcca-e7bd-4553-818f-66ee57939216 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.463247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cadcca-e7bd-4553-818f-66ee57939216 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.463525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cadcca-e7bd-4553-818f-66ee57939216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.487036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d9cadcca-e7bd-4553-818f-66ee57939216 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:08.488332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d9cadcca-e7bd-4553-818f-66ee57939216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:08.590449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 220/436] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:08.614820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0da41-a767-45d6-b6a7-9738a895f0c3 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:08.653935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570e446-935d-4d9e-b669-f65d046e32c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:08.654846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570e446-935d-4d9e-b669-f65d046e32c1 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.655128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570e446-935d-4d9e-b669-f65d046e32c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.655397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570e446-935d-4d9e-b669-f65d046e32c1 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.656059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570e446-935d-4d9e-b669-f65d046e32c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:08.656357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570e446-935d-4d9e-b669-f65d046e32c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:08.657761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570e446-935d-4d9e-b669-f65d046e32c1 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:08.668150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 217/437] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:17:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:08.674927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cadcca-e7bd-4553-818f-66ee57939216 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:08.766783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:08.798050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cadcca-e7bd-4553-818f-66ee57939216 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:08.826256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3d8f7-4133-4c9c-8122-f8bb3d27c1e9 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.826570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3d8f7-4133-4c9c-8122-f8bb3d27c1e9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.826809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3d8f7-4133-4c9c-8122-f8bb3d27c1e9 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.828240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 218/438] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:08.834774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512d9ff-291f-4ae7-a5b1-9cb74bf6ba9b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.835220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512d9ff-291f-4ae7-a5b1-9cb74bf6ba9b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.835613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512d9ff-291f-4ae7-a5b1-9cb74bf6ba9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.839047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50b7d-29d7-4603-9471-729be9a890d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:08.839654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50b7d-29d7-4603-9471-729be9a890d2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:08.839977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50b7d-29d7-4603-9471-729be9a890d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:08.840257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50b7d-29d7-4603-9471-729be9a890d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:08.861521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b512d9ff-291f-4ae7-a5b1-9cb74bf6ba9b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:08.862722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b512d9ff-291f-4ae7-a5b1-9cb74bf6ba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:08.924683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce50b7d-29d7-4603-9471-729be9a890d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:08.926708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce50b7d-29d7-4603-9471-729be9a890d2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:08.932479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:08.946078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 219/439] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:08.947278 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512d9ff-291f-4ae7-a5b1-9cb74bf6ba9b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.027250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:09.057288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 221/440] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:09.062521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512d9ff-291f-4ae7-a5b1-9cb74bf6ba9b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.062889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50b7d-29d7-4603-9471-729be9a890d2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.099878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f76aaf-dc8f-4aa6-8fad-6e7c856a41c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.100522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f76aaf-dc8f-4aa6-8fad-6e7c856a41c1 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.101057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f76aaf-dc8f-4aa6-8fad-6e7c856a41c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.101325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f76aaf-dc8f-4aa6-8fad-6e7c856a41c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.111618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f76aaf-dc8f-4aa6-8fad-6e7c856a41c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-f2f76aaf-dc8f-4aa6-8fad-6e7c856a41c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:09.117209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:09.118075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:09.118889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 220/441] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:09.123826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f76aaf-dc8f-4aa6-8fad-6e7c856a41c1 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.168366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50b7d-29d7-4603-9471-729be9a890d2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.184481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:09.204012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6de9-17da-4f9a-b419-43d77be0420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.204675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6de9-17da-4f9a-b419-43d77be0420c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.204975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6de9-17da-4f9a-b419-43d77be0420c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.205304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6de9-17da-4f9a-b419-43d77be0420c None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.207039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d6de9-17da-4f9a-b419-43d77be0420c None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:09.207925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d6de9-17da-4f9a-b419-43d77be0420c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:09.209737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 222/442] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:09.215005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6de9-17da-4f9a-b419-43d77be0420c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.240991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f76aaf-dc8f-4aa6-8fad-6e7c856a41c1 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.263270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:09.274563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfb5c1-9cb7-47c5-823d-00d3578e936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.275162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfb5c1-9cb7-47c5-823d-00d3578e936a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.275438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfb5c1-9cb7-47c5-823d-00d3578e936a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.275706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfb5c1-9cb7-47c5-823d-00d3578e936a None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.276359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfb5c1-9cb7-47c5-823d-00d3578e936a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:09.276685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfb5c1-9cb7-47c5-823d-00d3578e936a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:09.278113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfb5c1-9cb7-47c5-823d-00d3578e936a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:09.286507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 221/443] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:09.287275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfb5c1-9cb7-47c5-823d-00d3578e936a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.320897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6de9-17da-4f9a-b419-43d77be0420c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.353823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b9548-3ddb-437b-bb39-9adf9f782618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.354440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b9548-3ddb-437b-bb39-9adf9f782618 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.354718 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b9548-3ddb-437b-bb39-9adf9f782618 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.354990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b9548-3ddb-437b-bb39-9adf9f782618 None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.356108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b9548-3ddb-437b-bb39-9adf9f782618 None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:09.357584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b9548-3ddb-437b-bb39-9adf9f782618 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:09.359127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 223/444] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:09.361375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b9548-3ddb-437b-bb39-9adf9f782618 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.387656 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfb5c1-9cb7-47c5-823d-00d3578e936a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.401451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:09.422241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe54115-bfed-42dc-aba5-346d8229b935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.422857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe54115-bfed-42dc-aba5-346d8229b935 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.423132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe54115-bfed-42dc-aba5-346d8229b935 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.423406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe54115-bfed-42dc-aba5-346d8229b935 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.433310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe54115-bfed-42dc-aba5-346d8229b935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-ebe54115-bfed-42dc-aba5-346d8229b935 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:09.438491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:09.439389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:09.440199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 222/445] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:09.440892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe54115-bfed-42dc-aba5-346d8229b935 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.462723 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b9548-3ddb-437b-bb39-9adf9f782618 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.473322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:09.499713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487bc17-bb53-408f-a500-66bb1b6dfc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.500340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487bc17-bb53-408f-a500-66bb1b6dfc6d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.500650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487bc17-bb53-408f-a500-66bb1b6dfc6d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.500924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487bc17-bb53-408f-a500-66bb1b6dfc6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.545100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe54115-bfed-42dc-aba5-346d8229b935 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.570686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487bc17-bb53-408f-a500-66bb1b6dfc6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:09.572132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487bc17-bb53-408f-a500-66bb1b6dfc6d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:09.579123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d1500-4b79-4adf-a6cb-2fc1bd07193a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.579711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d1500-4b79-4adf-a6cb-2fc1bd07193a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.579984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d1500-4b79-4adf-a6cb-2fc1bd07193a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.580323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d1500-4b79-4adf-a6cb-2fc1bd07193a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.581077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d1500-4b79-4adf-a6cb-2fc1bd07193a None admin] RBAC: Adding query filter params (group.id=dbf5dbce88cc45ea82cb7056708a2091, role.id=df2ecb74b9fe42089e380b16de5ad950, scope.project.id=9a8fffb476e54c20b9662b528c1868ba) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:09.581386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d1500-4b79-4adf-a6cb-2fc1bd07193a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:09.583309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d1500-4b79-4adf-a6cb-2fc1bd07193a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:09.588872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 223/446] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/role_assignments?group.id=dbf5dbce88cc45ea82cb7056708a2091&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=df2ecb74b9fe42089e380b16de5ad950 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:09.589679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d1500-4b79-4adf-a6cb-2fc1bd07193a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.640958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:09.700762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 224/447] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:09.706637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487bc17-bb53-408f-a500-66bb1b6dfc6d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.717815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d1500-4b79-4adf-a6cb-2fc1bd07193a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.719582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:09.764669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f936b-f7b1-427b-ae86-8f185b72189a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.765523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f936b-f7b1-427b-ae86-8f185b72189a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.767730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f936b-f7b1-427b-ae86-8f185b72189a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.767730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f936b-f7b1-427b-ae86-8f185b72189a None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.767730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f936b-f7b1-427b-ae86-8f185b72189a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:09.767730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f936b-f7b1-427b-ae86-8f185b72189a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:09.768483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f936b-f7b1-427b-ae86-8f185b72189a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:09.777810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 224/448] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:09.783645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f936b-f7b1-427b-ae86-8f185b72189a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.826085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487bc17-bb53-408f-a500-66bb1b6dfc6d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.871201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f50d3-0a5c-41cb-89c4-54477807a725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.871840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f50d3-0a5c-41cb-89c4-54477807a725 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.872119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f50d3-0a5c-41cb-89c4-54477807a725 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.872393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f50d3-0a5c-41cb-89c4-54477807a725 None admin] PATH_INFO: `/v3/projects/f6a8f68ec23146108279fe93631f855a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.873632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f50d3-0a5c-41cb-89c4-54477807a725 None admin] RBAC: Authorizing `identity:get_project(project_id=f6a8f68ec23146108279fe93631f855a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:09.875136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f50d3-0a5c-41cb-89c4-54477807a725 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:09.881499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 225/449] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/projects/f6a8f68ec23146108279fe93631f855a => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:09.882627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f50d3-0a5c-41cb-89c4-54477807a725 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:09.901881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:09.905135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f936b-f7b1-427b-ae86-8f185b72189a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:09.951552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c04cd-c9d2-4991-8f4e-e6bed85e1d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:09.952159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c04cd-c9d2-4991-8f4e-e6bed85e1d6f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:09.952431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c04cd-c9d2-4991-8f4e-e6bed85e1d6f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:09.952750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c04cd-c9d2-4991-8f4e-e6bed85e1d6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:09.970896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:09.993070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f50d3-0a5c-41cb-89c4-54477807a725 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:10.020289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c04cd-c9d2-4991-8f4e-e6bed85e1d6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:10.021803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c04cd-c9d2-4991-8f4e-e6bed85e1d6f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:10.026162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a1b85-9ef0-42c3-b5ec-b0540a8cfbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:10.026784 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a1b85-9ef0-42c3-b5ec-b0540a8cfbd4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:10.027055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a1b85-9ef0-42c3-b5ec-b0540a8cfbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:10.027324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a1b85-9ef0-42c3-b5ec-b0540a8cfbd4 None admin] PATH_INFO: `/v3/groups/690efee7c72c4dc2a23769e57cf65253` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:10.028449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a1b85-9ef0-42c3-b5ec-b0540a8cfbd4 None admin] RBAC: Authorizing `identity:get_group(group_id=690efee7c72c4dc2a23769e57cf65253)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:10.029899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a1b85-9ef0-42c3-b5ec-b0540a8cfbd4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:10.031739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 226/450] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/groups/690efee7c72c4dc2a23769e57cf65253 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:10.037574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a1b85-9ef0-42c3-b5ec-b0540a8cfbd4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:10.115192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:10.123756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 225/451] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 00:17:10.124593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c04cd-c9d2-4991-8f4e-e6bed85e1d6f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:10.136489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a1b85-9ef0-42c3-b5ec-b0540a8cfbd4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:10.169470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b88cc9-5cda-4b9f-a919-f5318374e67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:10.170058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b88cc9-5cda-4b9f-a919-f5318374e67e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:10.170336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b88cc9-5cda-4b9f-a919-f5318374e67e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:10.170607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b88cc9-5cda-4b9f-a919-f5318374e67e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:10.171325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b88cc9-5cda-4b9f-a919-f5318374e67e None admin] RBAC: Adding query filter params (group.id=690efee7c72c4dc2a23769e57cf65253, role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=f6a8f68ec23146108279fe93631f855a) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:10.171630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b88cc9-5cda-4b9f-a919-f5318374e67e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:10.173098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b88cc9-5cda-4b9f-a919-f5318374e67e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:10.179334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 227/452] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:17:10 2022] GET /identity/v3/role_assignments?group.id=690efee7c72c4dc2a23769e57cf65253&scope.project.id=f6a8f68ec23146108279fe93631f855a&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:10.225400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c04cd-c9d2-4991-8f4e-e6bed85e1d6f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:10.264680 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f6387-591e-481c-b03b-cc904ef450e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:10.265333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f6387-591e-481c-b03b-cc904ef450e4 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:10.265575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f6387-591e-481c-b03b-cc904ef450e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:10.265865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f6387-591e-481c-b03b-cc904ef450e4 None admin] PATH_INFO: `/v3/projects/9a8fffb476e54c20b9662b528c1868ba` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:10.267016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f6387-591e-481c-b03b-cc904ef450e4 None admin] RBAC: Authorizing `identity:get_project(project_id=9a8fffb476e54c20b9662b528c1868ba)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:10.268516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f6387-591e-481c-b03b-cc904ef450e4 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:10.271206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 226/453] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:17:10 2022] GET /identity/v3/projects/9a8fffb476e54c20b9662b528c1868ba => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:10.276968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b88cc9-5cda-4b9f-a919-f5318374e67e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:10.291025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:10.382011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b88cc9-5cda-4b9f-a919-f5318374e67e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:10.414830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a0300-5721-4bd9-8412-1d51cef72f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:10.415450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a0300-5721-4bd9-8412-1d51cef72f42 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:10.415733 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a0300-5721-4bd9-8412-1d51cef72f42 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:10.416005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a0300-5721-4bd9-8412-1d51cef72f42 None admin] PATH_INFO: `/v3/groups/dbf5dbce88cc45ea82cb7056708a2091` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:10.417093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a0300-5721-4bd9-8412-1d51cef72f42 None admin] RBAC: Authorizing `identity:get_group(group_id=dbf5dbce88cc45ea82cb7056708a2091)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:10.418535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742a0300-5721-4bd9-8412-1d51cef72f42 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:10.420101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 228/454] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 1 00:17:10 2022] GET /identity/v3/groups/dbf5dbce88cc45ea82cb7056708a2091 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:10.426225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f6387-591e-481c-b03b-cc904ef450e4 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:10.490783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:10.523777 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f6387-591e-481c-b03b-cc904ef450e4 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:10.555583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de89a89-9d4d-4987-8cbd-691cf2fac322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:10.556165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de89a89-9d4d-4987-8cbd-691cf2fac322 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:10.556441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de89a89-9d4d-4987-8cbd-691cf2fac322 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:10.556737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de89a89-9d4d-4987-8cbd-691cf2fac322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:10.557453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de89a89-9d4d-4987-8cbd-691cf2fac322 None admin] RBAC: Adding query filter params (group.id=dbf5dbce88cc45ea82cb7056708a2091, role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=9a8fffb476e54c20b9662b528c1868ba) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:10.557760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de89a89-9d4d-4987-8cbd-691cf2fac322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:10.559182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de89a89-9d4d-4987-8cbd-691cf2fac322 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:10.564708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 227/455] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 1 00:17:10 2022] GET /identity/v3/role_assignments?group.id=dbf5dbce88cc45ea82cb7056708a2091&scope.project.id=9a8fffb476e54c20b9662b528c1868ba&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:12.938357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b125-874f-4798-bad5-bd35099eb1cf None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:12.939371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b125-874f-4798-bad5-bd35099eb1cf None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:12.939844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052b125-874f-4798-bad5-bd35099eb1cf None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:12.942240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 229/456] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:12.947565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98213-c525-4a2f-834a-13366ea10d92 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:12.948442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98213-c525-4a2f-834a-13366ea10d92 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:12.948966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98213-c525-4a2f-834a-13366ea10d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:12.973892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-62d98213-c525-4a2f-834a-13366ea10d92 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:12.975441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-62d98213-c525-4a2f-834a-13366ea10d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:13.058474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 228/457] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:13.190732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0229254-1f95-41b1-ad7c-d792e42f9e7a None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:13.191905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0229254-1f95-41b1-ad7c-d792e42f9e7a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:13.192473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0229254-1f95-41b1-ad7c-d792e42f9e7a None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:13.194303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 230/458] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:13.199112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cc04a-32d0-4612-ba70-9f6ea72dfe94 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:13.199620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cc04a-32d0-4612-ba70-9f6ea72dfe94 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:13.200108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cc04a-32d0-4612-ba70-9f6ea72dfe94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:13.221612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-836cc04a-32d0-4612-ba70-9f6ea72dfe94 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:13.223116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-836cc04a-32d0-4612-ba70-9f6ea72dfe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:13.313006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:13.324101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 229/459] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:13.341410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a316e7-f253-4d6e-b03a-f25d9627be1f None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:13.341953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a316e7-f253-4d6e-b03a-f25d9627be1f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:13.342436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a316e7-f253-4d6e-b03a-f25d9627be1f None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:13.343964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 231/460] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:13.349281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800caae-c8cd-4147-a73c-dfa7900f51f9 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:13.349812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800caae-c8cd-4147-a73c-dfa7900f51f9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:13.350289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800caae-c8cd-4147-a73c-dfa7900f51f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:13.376756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0800caae-c8cd-4147-a73c-dfa7900f51f9 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:13.378242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0800caae-c8cd-4147-a73c-dfa7900f51f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:13.384606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a316e7-f253-4d6e-b03a-f25d9627be1f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:13.504938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:13.510142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 230/461] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:13.538470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a316e7-f253-4d6e-b03a-f25d9627be1f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:13.601799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4e8e2-6d08-44d6-aa5b-2dfb00036477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:13.602791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4e8e2-6d08-44d6-aa5b-2dfb00036477 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:13.603268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4e8e2-6d08-44d6-aa5b-2dfb00036477 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:13.603760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4e8e2-6d08-44d6-aa5b-2dfb00036477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:13.620049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4e8e2-6d08-44d6-aa5b-2dfb00036477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:13.621803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4e8e2-6d08-44d6-aa5b-2dfb00036477 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-5ad4e8e2-6d08-44d6-aa5b-2dfb00036477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:13.643739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:13.644747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:13.645624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:13.645624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:13.645624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:13.645624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 232/462] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:13.650271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800caae-c8cd-4147-a73c-dfa7900f51f9 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:13.654373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4e8e2-6d08-44d6-aa5b-2dfb00036477 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:13.766654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4e8e2-6d08-44d6-aa5b-2dfb00036477 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:13.802214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:13.806964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263a5db-5dd7-4aa9-9237-10f141d79764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:13.807572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263a5db-5dd7-4aa9-9237-10f141d79764 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:13.807845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263a5db-5dd7-4aa9-9237-10f141d79764 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:13.808113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263a5db-5dd7-4aa9-9237-10f141d79764 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:13.808781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263a5db-5dd7-4aa9-9237-10f141d79764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:13.809249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263a5db-5dd7-4aa9-9237-10f141d79764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:13.810960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263a5db-5dd7-4aa9-9237-10f141d79764 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:13.833047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 233/463] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:13.845475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263a5db-5dd7-4aa9-9237-10f141d79764 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:13.886049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800caae-c8cd-4147-a73c-dfa7900f51f9 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:13.911689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:13.923336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd9d72-8d0f-43bb-bf17-3fa860a42e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:13.923976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd9d72-8d0f-43bb-bf17-3fa860a42e92 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:13.924257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd9d72-8d0f-43bb-bf17-3fa860a42e92 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:13.924553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd9d72-8d0f-43bb-bf17-3fa860a42e92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:13.935830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd9d72-8d0f-43bb-bf17-3fa860a42e92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:13.937443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd9d72-8d0f-43bb-bf17-3fa860a42e92 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-1cbd9d72-8d0f-43bb-bf17-3fa860a42e92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:13.953638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:13.955146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:13.955990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:13.955990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:13.955990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:13.955990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 231/464] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:13.955990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd9d72-8d0f-43bb-bf17-3fa860a42e92 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:13.962651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263a5db-5dd7-4aa9-9237-10f141d79764 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:13.998626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc468d30-c86f-4b61-a6ab-8cfff4e2784e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:13.999340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc468d30-c86f-4b61-a6ab-8cfff4e2784e None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:13.999612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc468d30-c86f-4b61-a6ab-8cfff4e2784e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:13.999878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc468d30-c86f-4b61-a6ab-8cfff4e2784e None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.000659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc468d30-c86f-4b61-a6ab-8cfff4e2784e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:14.002076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc468d30-c86f-4b61-a6ab-8cfff4e2784e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:14.026511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 234/465] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:17:13 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:14.027710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd8898-c14a-4ba3-986c-78f1bdb369bd None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.027990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd8898-c14a-4ba3-986c-78f1bdb369bd None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.028316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd8898-c14a-4ba3-986c-78f1bdb369bd None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.032664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 235/466] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:14.032664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd8898-c14a-4ba3-986c-78f1bdb369bd None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:14.073217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd9d72-8d0f-43bb-bf17-3fa860a42e92 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:14.085851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:14.125232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809ea1b-1320-412c-aa59-c3907b2e2805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:14.126561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809ea1b-1320-412c-aa59-c3907b2e2805 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.126954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809ea1b-1320-412c-aa59-c3907b2e2805 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.127323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809ea1b-1320-412c-aa59-c3907b2e2805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.141433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809ea1b-1320-412c-aa59-c3907b2e2805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:14.143399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809ea1b-1320-412c-aa59-c3907b2e2805 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:14.168357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd8898-c14a-4ba3-986c-78f1bdb369bd None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-6809ea1b-1320-412c-aa59-c3907b2e2805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:14.168655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:14.169536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:14.170371 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:14.170371 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:14.170371 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:14.172797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 232/467] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:14.174486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a70ec-ce19-4b5f-89ad-afb850eaeeb6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.175004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a70ec-ce19-4b5f-89ad-afb850eaeeb6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.175471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a70ec-ce19-4b5f-89ad-afb850eaeeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.203254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-752a70ec-ce19-4b5f-89ad-afb850eaeeb6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:14.204519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-752a70ec-ce19-4b5f-89ad-afb850eaeeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:14.213375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281dcd9-1ba6-42cb-85a9-04ad62d6f967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:14.213987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281dcd9-1ba6-42cb-85a9-04ad62d6f967 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.214273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281dcd9-1ba6-42cb-85a9-04ad62d6f967 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.214538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281dcd9-1ba6-42cb-85a9-04ad62d6f967 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.215183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281dcd9-1ba6-42cb-85a9-04ad62d6f967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:14.215480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281dcd9-1ba6-42cb-85a9-04ad62d6f967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:14.216901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281dcd9-1ba6-42cb-85a9-04ad62d6f967 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:14.229847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:14.233953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 236/468] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:14.235739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281dcd9-1ba6-42cb-85a9-04ad62d6f967 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:14.288422 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 233/469] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:14.289519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a70ec-ce19-4b5f-89ad-afb850eaeeb6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:14.373790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281dcd9-1ba6-42cb-85a9-04ad62d6f967 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:14.387114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:14.412022 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a70ec-ce19-4b5f-89ad-afb850eaeeb6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:14.420936 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38cfaf-16fd-487e-b66b-b49fcf7676f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:14.421558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38cfaf-16fd-487e-b66b-b49fcf7676f7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.421838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38cfaf-16fd-487e-b66b-b49fcf7676f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.422177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38cfaf-16fd-487e-b66b-b49fcf7676f7 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.423245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38cfaf-16fd-487e-b66b-b49fcf7676f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:14.423553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38cfaf-16fd-487e-b66b-b49fcf7676f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:14.425740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38cfaf-16fd-487e-b66b-b49fcf7676f7 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:14.442542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 237/470] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:14.449487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38cfaf-16fd-487e-b66b-b49fcf7676f7 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:14.457174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f2650-c586-4f0c-a3ab-6ce76e17047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:14.457873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f2650-c586-4f0c-a3ab-6ce76e17047f None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.458051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f2650-c586-4f0c-a3ab-6ce76e17047f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.458319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f2650-c586-4f0c-a3ab-6ce76e17047f None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.458955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f2650-c586-4f0c-a3ab-6ce76e17047f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:14.460376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f2650-c586-4f0c-a3ab-6ce76e17047f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:14.512641 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 234/471] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:17:14 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:14.597408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f2650-c586-4f0c-a3ab-6ce76e17047f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:14.599504 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38cfaf-16fd-487e-b66b-b49fcf7676f7 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:14.615741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:14.659621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39838b-f6c9-4da0-820c-a44eec89e215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:14.661567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39838b-f6c9-4da0-820c-a44eec89e215 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.662049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39838b-f6c9-4da0-820c-a44eec89e215 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.663478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39838b-f6c9-4da0-820c-a44eec89e215 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.664365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39838b-f6c9-4da0-820c-a44eec89e215 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:14.667026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39838b-f6c9-4da0-820c-a44eec89e215 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:14.703843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 238/472] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:17:14 2022] POST /identity/v3/users => generated 248 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:14.751835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f2650-c586-4f0c-a3ab-6ce76e17047f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:14.791365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db111b5a-ec74-45ee-95fb-effdf18f49d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:14.792023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db111b5a-ec74-45ee-95fb-effdf18f49d5 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.792291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db111b5a-ec74-45ee-95fb-effdf18f49d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.792585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db111b5a-ec74-45ee-95fb-effdf18f49d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.807478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db111b5a-ec74-45ee-95fb-effdf18f49d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:14.809136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db111b5a-ec74-45ee-95fb-effdf18f49d5 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-db111b5a-ec74-45ee-95fb-effdf18f49d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:14.826310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:14.827250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:14.828068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:14.828068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:14.828068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:14.828068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 235/473] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:14.835034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39838b-f6c9-4da0-820c-a44eec89e215 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:14.855837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce35b6-19d4-4999-a081-c647d1206099 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.856009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce35b6-19d4-4999-a081-c647d1206099 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.856274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce35b6-19d4-4999-a081-c647d1206099 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.857974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 236/474] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:14.864659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf1382-2be6-45ca-a9e9-6a15ef762730 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:14.864659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf1382-2be6-45ca-a9e9-6a15ef762730 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:14.864907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf1382-2be6-45ca-a9e9-6a15ef762730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:14.868366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:14.890673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-d6bf1382-2be6-45ca-a9e9-6a15ef762730 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:14.891883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d6bf1382-2be6-45ca-a9e9-6a15ef762730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:14.963333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:14.972959 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 237/475] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:14.973662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39838b-f6c9-4da0-820c-a44eec89e215 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:15.013792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bfc14-8089-4d74-9d2e-fbe3b2a95635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:15.014416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bfc14-8089-4d74-9d2e-fbe3b2a95635 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:15.014701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bfc14-8089-4d74-9d2e-fbe3b2a95635 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:15.014980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bfc14-8089-4d74-9d2e-fbe3b2a95635 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:15.015699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bfc14-8089-4d74-9d2e-fbe3b2a95635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:15.016012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bfc14-8089-4d74-9d2e-fbe3b2a95635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:15.017449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bfc14-8089-4d74-9d2e-fbe3b2a95635 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:15.030760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 239/476] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:15.036669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf1382-2be6-45ca-a9e9-6a15ef762730 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:15.148658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf1382-2be6-45ca-a9e9-6a15ef762730 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:15.185082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5bd12-e347-4d9e-a4ff-72d9eefdaf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:15.185699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5bd12-e347-4d9e-a4ff-72d9eefdaf6d None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:15.185977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5bd12-e347-4d9e-a4ff-72d9eefdaf6d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:15.186250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5bd12-e347-4d9e-a4ff-72d9eefdaf6d None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:15.187272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5bd12-e347-4d9e-a4ff-72d9eefdaf6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:15.188744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5bd12-e347-4d9e-a4ff-72d9eefdaf6d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:15.211926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 238/477] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:17:15 2022] POST /identity/v3/users => generated 249 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:15.274463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bfc14-8089-4d74-9d2e-fbe3b2a95635 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:15.366835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:15.387672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bfc14-8089-4d74-9d2e-fbe3b2a95635 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:15.425772 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:15.426376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:15.426656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:15.426926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:15.427613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:15.429035 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:15.438403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.sql.core [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 00:17:15.438403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 00:17:15.438403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [parameters: {'id': '8f734fbc5e8848119ac59b72695115ca', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 00:17:15.438403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93105) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application conn.query(q) Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application result.read() Sep 01 00:17:15.456011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 00:17:15.457064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application next(self.gen) Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application yield resource Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application next(self.gen) Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise self.value Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application session.commit() Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 00:17:15.457953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.session.flush() Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application compat.raise_( Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 00:17:15.458819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 00:17:15.459688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application conn.query(q) Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application result.read() Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 00:17:15.461760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [parameters: {'id': '8f734fbc5e8848119ac59b72695115ca', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:15.463763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 00:17:15.466049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:15.468283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 240/478] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 00:17:15 2022] POST /identity/v3/roles => generated 144 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:15.468423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5bd12-e347-4d9e-a4ff-72d9eefdaf6d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:15.522649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:15.577867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5bd12-e347-4d9e-a4ff-72d9eefdaf6d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:15.615057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707001f-f42e-4f28-b53b-cf773c7b4d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:15.615695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707001f-f42e-4f28-b53b-cf773c7b4d62 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:15.615973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707001f-f42e-4f28-b53b-cf773c7b4d62 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:15.616245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707001f-f42e-4f28-b53b-cf773c7b4d62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:15.626894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707001f-f42e-4f28-b53b-cf773c7b4d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-2707001f-f42e-4f28-b53b-cf773c7b4d62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:15.632574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:15.633424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:15.634232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 239/479] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:15.639605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:15.747363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a0209-9537-415e-995d-617917aa2ee3 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:15.783120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fe18c-7ccf-49b4-b593-437498075d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:15.783736 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fe18c-7ccf-49b4-b593-437498075d73 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:15.784014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fe18c-7ccf-49b4-b593-437498075d73 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:15.784290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fe18c-7ccf-49b4-b593-437498075d73 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:15.785140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fe18c-7ccf-49b4-b593-437498075d73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:15.785556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fe18c-7ccf-49b4-b593-437498075d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:15.786945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fe18c-7ccf-49b4-b593-437498075d73 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:15.795775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 241/480] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:16.499646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149b4a3-4f15-4f49-937c-0aac7cf8e3d0 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:16.500280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149b4a3-4f15-4f49-937c-0aac7cf8e3d0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:16.500280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149b4a3-4f15-4f49-937c-0aac7cf8e3d0 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:16.501850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 240/481] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:16.507289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92a38f-323b-47d1-a197-6c03ed6b7a05 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:16.507778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92a38f-323b-47d1-a197-6c03ed6b7a05 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:16.507965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92a38f-323b-47d1-a197-6c03ed6b7a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:16.529228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ce92a38f-323b-47d1-a197-6c03ed6b7a05 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:16.530406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ce92a38f-323b-47d1-a197-6c03ed6b7a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:16.532445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:16.607021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 242/482] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:17.004252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149b4a3-4f15-4f49-937c-0aac7cf8e3d0 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:17.060535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b35b3a-84d3-4375-b072-4e7667f218bc None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:17.060535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b35b3a-84d3-4375-b072-4e7667f218bc None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:17.060535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b35b3a-84d3-4375-b072-4e7667f218bc None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:17.062616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 243/483] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:17.068318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7c0e4-fe03-47d0-a66e-865a52fa651d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:17.068975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7c0e4-fe03-47d0-a66e-865a52fa651d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:17.069440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7c0e4-fe03-47d0-a66e-865a52fa651d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:17.087657 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-cae7c0e4-fe03-47d0-a66e-865a52fa651d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:17.089458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-cae7c0e4-fe03-47d0-a66e-865a52fa651d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:17.137590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149b4a3-4f15-4f49-937c-0aac7cf8e3d0 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:17.149539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:17.165830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 244/484] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:17.172619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e0ef6-09a5-4018-add4-8c1d8990a8fe None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:17.172619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e0ef6-09a5-4018-add4-8c1d8990a8fe None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:17.172619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e0ef6-09a5-4018-add4-8c1d8990a8fe None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:17.172619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 245/485] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:17.174347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f024e-1aa6-469e-8342-80d9f07a13e8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:17.174631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f024e-1aa6-469e-8342-80d9f07a13e8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:17.174905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f024e-1aa6-469e-8342-80d9f07a13e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:17.194666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a23f024e-1aa6-469e-8342-80d9f07a13e8 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:17.195627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7e8c-21d9-4669-a35e-b6a33a52a615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:17.195868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a23f024e-1aa6-469e-8342-80d9f07a13e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:17.196782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7e8c-21d9-4669-a35e-b6a33a52a615 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:17.197254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7e8c-21d9-4669-a35e-b6a33a52a615 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:17.197734 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7e8c-21d9-4669-a35e-b6a33a52a615 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:17.213767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d7e8c-21d9-4669-a35e-b6a33a52a615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-0f0d7e8c-21d9-4669-a35e-b6a33a52a615 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:17.224519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:17.225410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:17.226451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:17.229073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 241/486] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:17.234810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7e8c-21d9-4669-a35e-b6a33a52a615 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:17.304469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:17.330130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 246/487] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:17.350214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7e8c-21d9-4669-a35e-b6a33a52a615 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:17.385037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b96ff-85eb-423e-b0a0-862ded798180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:17.385966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b96ff-85eb-423e-b0a0-862ded798180 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:17.386455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b96ff-85eb-423e-b0a0-862ded798180 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:17.386918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b96ff-85eb-423e-b0a0-862ded798180 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:17.387762 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b96ff-85eb-423e-b0a0-862ded798180 None admin] RBAC: Adding query filter params (name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:17.388250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b96ff-85eb-423e-b0a0-862ded798180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:17.389820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b96ff-85eb-423e-b0a0-862ded798180 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:17.398560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 242/488] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:17.403770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f024e-1aa6-469e-8342-80d9f07a13e8 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:17.591754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f024e-1aa6-469e-8342-80d9f07a13e8 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:17.642188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff88d3-3010-4149-9d2a-7c9b25d6ddbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:17.642873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff88d3-3010-4149-9d2a-7c9b25d6ddbf None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:17.643152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff88d3-3010-4149-9d2a-7c9b25d6ddbf None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:17.643423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff88d3-3010-4149-9d2a-7c9b25d6ddbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:17.661157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff88d3-3010-4149-9d2a-7c9b25d6ddbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:17.662657 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff88d3-3010-4149-9d2a-7c9b25d6ddbf None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:17.666857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b96ff-85eb-423e-b0a0-862ded798180 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-8cff88d3-3010-4149-9d2a-7c9b25d6ddbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:17.688204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:17.689330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:17.690164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:17.690164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:17.690164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:17.690164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 247/489] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:17.695438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff88d3-3010-4149-9d2a-7c9b25d6ddbf None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:17.712921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:17.788994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:17.795124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b96ff-85eb-423e-b0a0-862ded798180 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:17.813065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff88d3-3010-4149-9d2a-7c9b25d6ddbf None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:17.836128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791a208-ddbd-4c1c-b576-1674c60b629c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:17.837572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791a208-ddbd-4c1c-b576-1674c60b629c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:17.838205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791a208-ddbd-4c1c-b576-1674c60b629c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:17.838785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791a208-ddbd-4c1c-b576-1674c60b629c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:17.851787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a550065-d5eb-4442-9b1e-9e266d2942fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:17.852328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a550065-d5eb-4442-9b1e-9e266d2942fd None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:17.852706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a550065-d5eb-4442-9b1e-9e266d2942fd None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:17.852992 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a550065-d5eb-4442-9b1e-9e266d2942fd None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:17.853422 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791a208-ddbd-4c1c-b576-1674c60b629c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:17.853645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a550065-d5eb-4442-9b1e-9e266d2942fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:17.853950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a550065-d5eb-4442-9b1e-9e266d2942fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:17.855340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a550065-d5eb-4442-9b1e-9e266d2942fd None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-f791a208-ddbd-4c1c-b576-1674c60b629c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:17.860185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:17.861085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:17.861888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:17.870401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 243/490] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:17.871816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791a208-ddbd-4c1c-b576-1674c60b629c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:17.876749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 248/491] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:17.877637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a550065-d5eb-4442-9b1e-9e266d2942fd None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:17.981115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:17.990523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791a208-ddbd-4c1c-b576-1674c60b629c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.008070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a550065-d5eb-4442-9b1e-9e266d2942fd None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.023089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:18.028811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ab20c-79ad-4845-a8d9-08bad8325201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.029446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ab20c-79ad-4845-a8d9-08bad8325201 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.029727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ab20c-79ad-4845-a8d9-08bad8325201 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.030038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ab20c-79ad-4845-a8d9-08bad8325201 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.041254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ab20c-79ad-4845-a8d9-08bad8325201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-8c3ab20c-79ad-4845-a8d9-08bad8325201 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:18.046669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:18.047530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:18.048335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 244/492] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:18.049511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ab20c-79ad-4845-a8d9-08bad8325201 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.056252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573608ab-4af8-415e-84b8-63837d9b167e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.057142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573608ab-4af8-415e-84b8-63837d9b167e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.057630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573608ab-4af8-415e-84b8-63837d9b167e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.058087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573608ab-4af8-415e-84b8-63837d9b167e None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.058986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573608ab-4af8-415e-84b8-63837d9b167e None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:18.059931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573608ab-4af8-415e-84b8-63837d9b167e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.061947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573608ab-4af8-415e-84b8-63837d9b167e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.073278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 249/493] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:18.074588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573608ab-4af8-415e-84b8-63837d9b167e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.181744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ab20c-79ad-4845-a8d9-08bad8325201 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.186710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573608ab-4af8-415e-84b8-63837d9b167e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.215827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92836c3-a1a3-496b-864a-39276f9d9f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.216446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92836c3-a1a3-496b-864a-39276f9d9f0c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.216751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92836c3-a1a3-496b-864a-39276f9d9f0c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.217020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92836c3-a1a3-496b-864a-39276f9d9f0c None admin] PATH_INFO: `/v3/users/nova` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.220856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d7e2-e562-4269-820e-35660522ad25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.221450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d7e2-e562-4269-820e-35660522ad25 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.221725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d7e2-e562-4269-820e-35660522ad25 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.221995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d7e2-e562-4269-820e-35660522ad25 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.222689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d7e2-e562-4269-820e-35660522ad25 None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:18.222987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d7e2-e562-4269-820e-35660522ad25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.223891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92836c3-a1a3-496b-864a-39276f9d9f0c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.224360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d7e2-e562-4269-820e-35660522ad25 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.225386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92836c3-a1a3-496b-864a-39276f9d9f0c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.232776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 250/494] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:18.233594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d7e2-e562-4269-820e-35660522ad25 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-e92836c3-a1a3-496b-864a-39276f9d9f0c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:18.240190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:18.241293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:18.242194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 245/495] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:18.245812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2f308-e6d5-4042-8ec5-f8df5cbdaade None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.246092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2f308-e6d5-4042-8ec5-f8df5cbdaade None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.246360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2f308-e6d5-4042-8ec5-f8df5cbdaade None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.247601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 246/496] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:18.251230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d11db-0fa5-42ac-9c15-2fa836f1f939 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.251230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d11db-0fa5-42ac-9c15-2fa836f1f939 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.251230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d11db-0fa5-42ac-9c15-2fa836f1f939 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.251230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 247/497] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:18.251230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d11db-0fa5-42ac-9c15-2fa836f1f939 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.266897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:18.274663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:18.341447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d7e2-e562-4269-820e-35660522ad25 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.351981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d11db-0fa5-42ac-9c15-2fa836f1f939 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.375944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e010d-fa13-4e0d-a5d0-0a090fd47394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.376604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e010d-fa13-4e0d-a5d0-0a090fd47394 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.376885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e010d-fa13-4e0d-a5d0-0a090fd47394 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.377164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e010d-fa13-4e0d-a5d0-0a090fd47394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.385290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2fa85-d049-40c4-9c82-129882170f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.385904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2fa85-d049-40c4-9c82-129882170f79 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.386178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2fa85-d049-40c4-9c82-129882170f79 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.386445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2fa85-d049-40c4-9c82-129882170f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.387909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e010d-fa13-4e0d-a5d0-0a090fd47394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.389427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e010d-fa13-4e0d-a5d0-0a090fd47394 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.396275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2fa85-d049-40c4-9c82-129882170f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.397764 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2fa85-d049-40c4-9c82-129882170f79 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-9d4e010d-fa13-4e0d-a5d0-0a090fd47394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:18.403385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:18.404286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:18.410488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:18.410488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:18.410488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:18.410488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 251/498] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:18.410488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e010d-fa13-4e0d-a5d0-0a090fd47394 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-1ee2fa85-d049-40c4-9c82-129882170f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:18.413257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:18.414275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:18.415113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:18.415113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:18.415113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:18.415113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 248/499] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:18.415769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370628f-165a-4cc7-af8b-ddbfd8d8b97c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.416049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370628f-165a-4cc7-af8b-ddbfd8d8b97c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.416319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370628f-165a-4cc7-af8b-ddbfd8d8b97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.434900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-8370628f-165a-4cc7-af8b-ddbfd8d8b97c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:18.436056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8370628f-165a-4cc7-af8b-ddbfd8d8b97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:18.494964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:18.496443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:18.504850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 249/500] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:18.506131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c895086-e582-4e72-9316-255757885fb6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.506419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c895086-e582-4e72-9316-255757885fb6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.506740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c895086-e582-4e72-9316-255757885fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.510469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e010d-fa13-4e0d-a5d0-0a090fd47394 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.524298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2c895086-e582-4e72-9316-255757885fb6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:18.525517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2c895086-e582-4e72-9316-255757885fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:18.546178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23116689-0b63-4923-b5de-544d91676d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.546786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23116689-0b63-4923-b5de-544d91676d02 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.547069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23116689-0b63-4923-b5de-544d91676d02 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.547340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23116689-0b63-4923-b5de-544d91676d02 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.548072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23116689-0b63-4923-b5de-544d91676d02 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:18.548383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23116689-0b63-4923-b5de-544d91676d02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.549843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23116689-0b63-4923-b5de-544d91676d02 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.586210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 252/501] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:18.587023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23116689-0b63-4923-b5de-544d91676d02 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.600131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 250/502] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:18.601491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c895086-e582-4e72-9316-255757885fb6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.704181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23116689-0b63-4923-b5de-544d91676d02 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.732129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c895086-e582-4e72-9316-255757885fb6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.751189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548d035-6f2a-4e2d-9abd-13379130f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.752192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548d035-6f2a-4e2d-9abd-13379130f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.752831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548d035-6f2a-4e2d-9abd-13379130f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.753320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548d035-6f2a-4e2d-9abd-13379130f1f4 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.755338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548d035-6f2a-4e2d-9abd-13379130f1f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:18.755823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548d035-6f2a-4e2d-9abd-13379130f1f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.757991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548d035-6f2a-4e2d-9abd-13379130f1f4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.758770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:18.770884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffb548-3b84-4240-8249-1260e07cf17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.771432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffb548-3b84-4240-8249-1260e07cf17b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.771712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffb548-3b84-4240-8249-1260e07cf17b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.771993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffb548-3b84-4240-8249-1260e07cf17b None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.772661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffb548-3b84-4240-8249-1260e07cf17b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:18.773049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffb548-3b84-4240-8249-1260e07cf17b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.774697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ffb548-3b84-4240-8249-1260e07cf17b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.775181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 253/503] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:18.775883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548d035-6f2a-4e2d-9abd-13379130f1f4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.780294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:18.790749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 251/504] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:18.790749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffb548-3b84-4240-8249-1260e07cf17b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.882291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548d035-6f2a-4e2d-9abd-13379130f1f4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.898120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffb548-3b84-4240-8249-1260e07cf17b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:18.915260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dc23c-7cac-499a-9dba-2c0a441f5d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.915857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dc23c-7cac-499a-9dba-2c0a441f5d1e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.916131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dc23c-7cac-499a-9dba-2c0a441f5d1e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.916396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dc23c-7cac-499a-9dba-2c0a441f5d1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.931557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82a4c9-7af1-47d3-8817-515ea40339a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:18.932168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82a4c9-7af1-47d3-8817-515ea40339a8 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:18.932439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82a4c9-7af1-47d3-8817-515ea40339a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:18.932732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82a4c9-7af1-47d3-8817-515ea40339a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:18.943515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82a4c9-7af1-47d3-8817-515ea40339a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.945005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82a4c9-7af1-47d3-8817-515ea40339a8 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-3d82a4c9-7af1-47d3-8817-515ea40339a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:18.959451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:18.960329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:18.961218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:18.961218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:18.961218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:18.961218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 252/505] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:18.961443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82a4c9-7af1-47d3-8817-515ea40339a8 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:18.985212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dc23c-7cac-499a-9dba-2c0a441f5d1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:18.986685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dc23c-7cac-499a-9dba-2c0a441f5d1e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:18.989616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:18.999701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:19.067397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82a4c9-7af1-47d3-8817-515ea40339a8 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.095991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 254/506] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:19.096870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dc23c-7cac-499a-9dba-2c0a441f5d1e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.099671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d58be-74ac-42b5-80b4-7bcc9cbc1c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.100289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d58be-74ac-42b5-80b4-7bcc9cbc1c7f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.100990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d58be-74ac-42b5-80b4-7bcc9cbc1c7f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.101269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d58be-74ac-42b5-80b4-7bcc9cbc1c7f None admin] PATH_INFO: `/v3/users/glance` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.108688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d58be-74ac-42b5-80b4-7bcc9cbc1c7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:19.110117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d58be-74ac-42b5-80b4-7bcc9cbc1c7f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-cf9d58be-74ac-42b5-80b4-7bcc9cbc1c7f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:19.121451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:19.122488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 253/507] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:19.123585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d58be-74ac-42b5-80b4-7bcc9cbc1c7f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.198393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dc23c-7cac-499a-9dba-2c0a441f5d1e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.200663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:19.217538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:19.223588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d58be-74ac-42b5-80b4-7bcc9cbc1c7f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.234768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f722e1c-800b-426d-b5c7-319864be8a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.235153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f722e1c-800b-426d-b5c7-319864be8a23 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.235431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f722e1c-800b-426d-b5c7-319864be8a23 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.235712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f722e1c-800b-426d-b5c7-319864be8a23 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.243530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f722e1c-800b-426d-b5c7-319864be8a23 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:19.245004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f722e1c-800b-426d-b5c7-319864be8a23 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-5f722e1c-800b-426d-b5c7-319864be8a23 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:19.258609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:19.259531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:19.260401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 255/508] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:19.260995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19588f5-5ca3-4761-8548-49b9bc776f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.261090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f722e1c-800b-426d-b5c7-319864be8a23 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.261158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19588f5-5ca3-4761-8548-49b9bc776f77 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.261158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19588f5-5ca3-4761-8548-49b9bc776f77 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.261315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19588f5-5ca3-4761-8548-49b9bc776f77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.274751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19588f5-5ca3-4761-8548-49b9bc776f77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-a19588f5-5ca3-4761-8548-49b9bc776f77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:19.280011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:19.281085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:19.282072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 254/509] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:19.282991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19588f5-5ca3-4761-8548-49b9bc776f77 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.367236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f722e1c-800b-426d-b5c7-319864be8a23 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.386574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19588f5-5ca3-4761-8548-49b9bc776f77 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.400886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a846e-b90c-45ad-93a7-742ff5ac0b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.401489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a846e-b90c-45ad-93a7-742ff5ac0b5d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.401760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a846e-b90c-45ad-93a7-742ff5ac0b5d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.402026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a846e-b90c-45ad-93a7-742ff5ac0b5d None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.402652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a846e-b90c-45ad-93a7-742ff5ac0b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:19.403047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a846e-b90c-45ad-93a7-742ff5ac0b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:19.404325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a846e-b90c-45ad-93a7-742ff5ac0b5d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:19.415805 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 256/510] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:19.416613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a846e-b90c-45ad-93a7-742ff5ac0b5d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.426408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f69a28-4861-4939-b042-183da0383cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.427178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f69a28-4861-4939-b042-183da0383cf6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.427450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f69a28-4861-4939-b042-183da0383cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.427718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f69a28-4861-4939-b042-183da0383cf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.438229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f69a28-4861-4939-b042-183da0383cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:19.439661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f69a28-4861-4939-b042-183da0383cf6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:19.440802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-94f69a28-4861-4939-b042-183da0383cf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:19.455860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:19.456921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:19.457994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:19.457994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:19.457994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:19.457994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 255/511] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:19.457994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f69a28-4861-4939-b042-183da0383cf6 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.469114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:19.518525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a846e-b90c-45ad-93a7-742ff5ac0b5d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.552303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a11ca6-49e7-4914-bd46-4d6ff6ddb53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.552918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a11ca6-49e7-4914-bd46-4d6ff6ddb53a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.553194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a11ca6-49e7-4914-bd46-4d6ff6ddb53a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.553464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a11ca6-49e7-4914-bd46-4d6ff6ddb53a None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.554224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a11ca6-49e7-4914-bd46-4d6ff6ddb53a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:19.554524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a11ca6-49e7-4914-bd46-4d6ff6ddb53a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:19.555954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a11ca6-49e7-4914-bd46-4d6ff6ddb53a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:19.557791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f69a28-4861-4939-b042-183da0383cf6 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.573758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 257/512] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:19.574553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a11ca6-49e7-4914-bd46-4d6ff6ddb53a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.596757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2e5a4-c4b4-403f-acaf-f705f94881dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.597387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2e5a4-c4b4-403f-acaf-f705f94881dc None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.597657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2e5a4-c4b4-403f-acaf-f705f94881dc None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.597948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2e5a4-c4b4-403f-acaf-f705f94881dc None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.598637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2e5a4-c4b4-403f-acaf-f705f94881dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:19.598935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2e5a4-c4b4-403f-acaf-f705f94881dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:19.600356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2e5a4-c4b4-403f-acaf-f705f94881dc None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:19.637669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 256/513] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:19.638461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2e5a4-c4b4-403f-acaf-f705f94881dc None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.659905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:19.680419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a11ca6-49e7-4914-bd46-4d6ff6ddb53a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.703547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:19.717620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3519817-b153-48c6-b25f-faea6730be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.718211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3519817-b153-48c6-b25f-faea6730be95 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.718485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3519817-b153-48c6-b25f-faea6730be95 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.718755 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3519817-b153-48c6-b25f-faea6730be95 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.719455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3519817-b153-48c6-b25f-faea6730be95 None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:19.719706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3519817-b153-48c6-b25f-faea6730be95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:19.721122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3519817-b153-48c6-b25f-faea6730be95 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:19.729747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 258/514] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:19.730526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3519817-b153-48c6-b25f-faea6730be95 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.743949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2e5a4-c4b4-403f-acaf-f705f94881dc None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.777629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab33f74-3302-449a-b829-db7f0a941d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.778234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab33f74-3302-449a-b829-db7f0a941d00 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.778512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab33f74-3302-449a-b829-db7f0a941d00 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.778791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab33f74-3302-449a-b829-db7f0a941d00 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.779429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab33f74-3302-449a-b829-db7f0a941d00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:19.780915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab33f74-3302-449a-b829-db7f0a941d00 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:19.802594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 257/515] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:17:19 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:19.803406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab33f74-3302-449a-b829-db7f0a941d00 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.837256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3519817-b153-48c6-b25f-faea6730be95 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.877321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2812f62-2f6f-45ec-89ab-991768789cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.877946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2812f62-2f6f-45ec-89ab-991768789cd6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.878240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2812f62-2f6f-45ec-89ab-991768789cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.879097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2812f62-2f6f-45ec-89ab-991768789cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:19.879929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2812f62-2f6f-45ec-89ab-991768789cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:19.880237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2812f62-2f6f-45ec-89ab-991768789cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:19.881709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2812f62-2f6f-45ec-89ab-991768789cd6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:19.895265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 259/516] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:19.895758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2812f62-2f6f-45ec-89ab-991768789cd6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:19.899569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:19.930447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab33f74-3302-449a-b829-db7f0a941d00 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:19.959515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:19.975089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397e6e8-9ac2-46dc-bbc8-d00cc1190c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:19.975960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397e6e8-9ac2-46dc-bbc8-d00cc1190c28 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:19.976233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397e6e8-9ac2-46dc-bbc8-d00cc1190c28 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:19.976549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397e6e8-9ac2-46dc-bbc8-d00cc1190c28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.018112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2812f62-2f6f-45ec-89ab-991768789cd6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:20.056386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b80a6d-7257-4acc-96e9-05ba10ef12a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:20.056811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397e6e8-9ac2-46dc-bbc8-d00cc1190c28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.057432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b80a6d-7257-4acc-96e9-05ba10ef12a2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:20.057848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b80a6d-7257-4acc-96e9-05ba10ef12a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:20.058253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b80a6d-7257-4acc-96e9-05ba10ef12a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.058359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397e6e8-9ac2-46dc-bbc8-d00cc1190c28 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.127840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b80a6d-7257-4acc-96e9-05ba10ef12a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.130406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b80a6d-7257-4acc-96e9-05ba10ef12a2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.133265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:20.164605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 258/517] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:20.165525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397e6e8-9ac2-46dc-bbc8-d00cc1190c28 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.190523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:20.249247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 260/518] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:20.249247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b80a6d-7257-4acc-96e9-05ba10ef12a2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.278070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397e6e8-9ac2-46dc-bbc8-d00cc1190c28 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:20.317212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e71a19-bba5-4797-addb-6cb272828985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:20.318270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e71a19-bba5-4797-addb-6cb272828985 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:20.318719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e71a19-bba5-4797-addb-6cb272828985 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:20.319160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e71a19-bba5-4797-addb-6cb272828985 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.331345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e71a19-bba5-4797-addb-6cb272828985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.333915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e71a19-bba5-4797-addb-6cb272828985 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-10e71a19-bba5-4797-addb-6cb272828985 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:20.352617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:20.353725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:20.354755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:20.354755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:20.354755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:20.356251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 259/519] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:20.357203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e71a19-bba5-4797-addb-6cb272828985 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.363811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b80a6d-7257-4acc-96e9-05ba10ef12a2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:20.366518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:20.400811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab1a46-b4f9-4f87-8a69-eac390b4bb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:20.401444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab1a46-b4f9-4f87-8a69-eac390b4bb34 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:20.401705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab1a46-b4f9-4f87-8a69-eac390b4bb34 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:20.401966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab1a46-b4f9-4f87-8a69-eac390b4bb34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.409350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab1a46-b4f9-4f87-8a69-eac390b4bb34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.411388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab1a46-b4f9-4f87-8a69-eac390b4bb34 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-ddab1a46-b4f9-4f87-8a69-eac390b4bb34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:20.423133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:20.423973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:20.424835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:20.424835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:20.424835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:20.424835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:20.424835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:20.424835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:20.424835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:20.424835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:20.424835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 261/520] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:20.426416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab1a46-b4f9-4f87-8a69-eac390b4bb34 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.441381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:20.467317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e71a19-bba5-4797-addb-6cb272828985 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:20.503914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d26ee-e62f-474f-aff3-2adb5317681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:20.504925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d26ee-e62f-474f-aff3-2adb5317681b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:20.505211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d26ee-e62f-474f-aff3-2adb5317681b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:20.505488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d26ee-e62f-474f-aff3-2adb5317681b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.517290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d26ee-e62f-474f-aff3-2adb5317681b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.519697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d26ee-e62f-474f-aff3-2adb5317681b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.532129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab1a46-b4f9-4f87-8a69-eac390b4bb34 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-866d26ee-e62f-474f-aff3-2adb5317681b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:20.534487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:20.535872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:20.537063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:20.537063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:20.537063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:20.537063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 260/521] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:20.537063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d26ee-e62f-474f-aff3-2adb5317681b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.573473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb88e4f-f992-4ea9-8290-00b19aa5425f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:20.574104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb88e4f-f992-4ea9-8290-00b19aa5425f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:20.574376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb88e4f-f992-4ea9-8290-00b19aa5425f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:20.574641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb88e4f-f992-4ea9-8290-00b19aa5425f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.585911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb88e4f-f992-4ea9-8290-00b19aa5425f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.587494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb88e4f-f992-4ea9-8290-00b19aa5425f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-4cb88e4f-f992-4ea9-8290-00b19aa5425f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:20.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:20.603869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:20.605143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:20.605143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:20.605143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:20.605143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 262/522] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:20.605143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb88e4f-f992-4ea9-8290-00b19aa5425f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.623901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:20.645566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d26ee-e62f-474f-aff3-2adb5317681b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:20.680156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:20.690833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a288de-70fa-48b3-8dfc-ae5050d981fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:20.691467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a288de-70fa-48b3-8dfc-ae5050d981fd None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:20.691741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a288de-70fa-48b3-8dfc-ae5050d981fd None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:20.692005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a288de-70fa-48b3-8dfc-ae5050d981fd None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.692678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a288de-70fa-48b3-8dfc-ae5050d981fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:20.692982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a288de-70fa-48b3-8dfc-ae5050d981fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.694366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a288de-70fa-48b3-8dfc-ae5050d981fd None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.707001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 261/523] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:20.708038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a288de-70fa-48b3-8dfc-ae5050d981fd None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.726019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb88e4f-f992-4ea9-8290-00b19aa5425f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:20.763195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243cce5-f8f8-4705-83f6-ffe4300f17ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:20.763816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243cce5-f8f8-4705-83f6-ffe4300f17ac None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:20.764151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243cce5-f8f8-4705-83f6-ffe4300f17ac None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:20.764428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243cce5-f8f8-4705-83f6-ffe4300f17ac None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.765152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243cce5-f8f8-4705-83f6-ffe4300f17ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:20.765453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243cce5-f8f8-4705-83f6-ffe4300f17ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.766885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243cce5-f8f8-4705-83f6-ffe4300f17ac None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.786004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 263/524] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:20.786924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243cce5-f8f8-4705-83f6-ffe4300f17ac None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.816593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a288de-70fa-48b3-8dfc-ae5050d981fd None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:20.853681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9ea2b-35d2-4cdd-98fd-35c9be9f7c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:20.854286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9ea2b-35d2-4cdd-98fd-35c9be9f7c1a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:20.854558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9ea2b-35d2-4cdd-98fd-35c9be9f7c1a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:20.854826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9ea2b-35d2-4cdd-98fd-35c9be9f7c1a None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.855461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9ea2b-35d2-4cdd-98fd-35c9be9f7c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:20.855756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9ea2b-35d2-4cdd-98fd-35c9be9f7c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.857154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9ea2b-35d2-4cdd-98fd-35c9be9f7c1a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.869331 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 262/525] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:20.870418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9ea2b-35d2-4cdd-98fd-35c9be9f7c1a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.876131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:20.901799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243cce5-f8f8-4705-83f6-ffe4300f17ac None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:20.922181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:20.939563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb048d39-06dd-4b84-b1ab-0b3c85f6b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:20.940174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb048d39-06dd-4b84-b1ab-0b3c85f6b38e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:20.940453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb048d39-06dd-4b84-b1ab-0b3c85f6b38e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:20.940759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb048d39-06dd-4b84-b1ab-0b3c85f6b38e None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:20.941621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb048d39-06dd-4b84-b1ab-0b3c85f6b38e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:20.941923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb048d39-06dd-4b84-b1ab-0b3c85f6b38e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:20.943328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb048d39-06dd-4b84-b1ab-0b3c85f6b38e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:20.955637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 264/526] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:20.956274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb048d39-06dd-4b84-b1ab-0b3c85f6b38e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:20.986719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9ea2b-35d2-4cdd-98fd-35c9be9f7c1a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.022533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ec068-7b1e-4134-a548-c0fcdc158d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.023075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ec068-7b1e-4134-a548-c0fcdc158d1c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.023427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ec068-7b1e-4134-a548-c0fcdc158d1c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.023630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ec068-7b1e-4134-a548-c0fcdc158d1c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.031384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ec068-7b1e-4134-a548-c0fcdc158d1c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.032861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ec068-7b1e-4134-a548-c0fcdc158d1c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-b90ec068-7b1e-4134-a548-c0fcdc158d1c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:21.045960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:21.046880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 263/527] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:21.053357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ec068-7b1e-4134-a548-c0fcdc158d1c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:21.070530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb048d39-06dd-4b84-b1ab-0b3c85f6b38e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.107168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e15ab-479e-49da-a0fd-d7359be07e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.107768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e15ab-479e-49da-a0fd-d7359be07e2d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.108038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e15ab-479e-49da-a0fd-d7359be07e2d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.108337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e15ab-479e-49da-a0fd-d7359be07e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.109089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e15ab-479e-49da-a0fd-d7359be07e2d None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=fb204040ab2f4168b35c3e78bacb849f) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:21.109377 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e15ab-479e-49da-a0fd-d7359be07e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.110842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e15ab-479e-49da-a0fd-d7359be07e2d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.114201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:21.117200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 265/528] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:20 2022] GET /identity/v3/role_assignments?user.id=fb204040ab2f4168b35c3e78bacb849f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:21.117967 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e15ab-479e-49da-a0fd-d7359be07e2d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:21.205729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ec068-7b1e-4134-a548-c0fcdc158d1c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.216009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:21.248629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e15ab-479e-49da-a0fd-d7359be07e2d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.267861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f5975-084a-4503-8bfe-d5a9efe7e106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.268535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f5975-084a-4503-8bfe-d5a9efe7e106 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.268802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f5975-084a-4503-8bfe-d5a9efe7e106 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.269158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f5975-084a-4503-8bfe-d5a9efe7e106 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.278690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f5975-084a-4503-8bfe-d5a9efe7e106 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.280110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f5975-084a-4503-8bfe-d5a9efe7e106 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-d58f5975-084a-4503-8bfe-d5a9efe7e106 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:21.297590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:21.298459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:21.299276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:21.299276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:21.299276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:21.299276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:21.299276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:21.299276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:21.299276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:21.299276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:21.299672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515a0d6-f288-48e5-a203-8c976c0bd7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.299672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515a0d6-f288-48e5-a203-8c976c0bd7f1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.299672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515a0d6-f288-48e5-a203-8c976c0bd7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.299672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515a0d6-f288-48e5-a203-8c976c0bd7f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.302527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 264/529] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:21.303995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f5975-084a-4503-8bfe-d5a9efe7e106 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:21.312529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515a0d6-f288-48e5-a203-8c976c0bd7f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.312529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515a0d6-f288-48e5-a203-8c976c0bd7f1 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-4515a0d6-f288-48e5-a203-8c976c0bd7f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:21.323746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:21.324836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:21.325884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:21.325884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:21.325884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:21.325884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:21.325884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:21.325884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:21.325884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:21.325884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:21.327004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 266/530] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:21.327796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515a0d6-f288-48e5-a203-8c976c0bd7f1 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:21.390837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:21.429191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f5975-084a-4503-8bfe-d5a9efe7e106 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.435121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515a0d6-f288-48e5-a203-8c976c0bd7f1 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.476325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092b02f-475d-41a2-a3e2-d99556307d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.476563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539c9be-ddc5-47f7-92c1-6d27080635f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.477104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092b02f-475d-41a2-a3e2-d99556307d5c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.477444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092b02f-475d-41a2-a3e2-d99556307d5c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.477652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539c9be-ddc5-47f7-92c1-6d27080635f1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.477831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092b02f-475d-41a2-a3e2-d99556307d5c None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.478208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539c9be-ddc5-47f7-92c1-6d27080635f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.478612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539c9be-ddc5-47f7-92c1-6d27080635f1 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.478729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092b02f-475d-41a2-a3e2-d99556307d5c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:21.479097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092b02f-475d-41a2-a3e2-d99556307d5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.479690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539c9be-ddc5-47f7-92c1-6d27080635f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:21.480128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539c9be-ddc5-47f7-92c1-6d27080635f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.480743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092b02f-475d-41a2-a3e2-d99556307d5c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.482006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539c9be-ddc5-47f7-92c1-6d27080635f1 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.498993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 267/531] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:21.500164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 265/532] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 1 00:17:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:21.500440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539c9be-ddc5-47f7-92c1-6d27080635f1 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:21.506152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092b02f-475d-41a2-a3e2-d99556307d5c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:21.532048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:21.616907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539c9be-ddc5-47f7-92c1-6d27080635f1 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.630610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092b02f-475d-41a2-a3e2-d99556307d5c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.641901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:21.659322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a06b8c-cc8a-4c82-9bd5-e010d0fdfac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.660286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06b8c-cc8a-4c82-9bd5-e010d0fdfac7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.660780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06b8c-cc8a-4c82-9bd5-e010d0fdfac7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.661366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a06b8c-cc8a-4c82-9bd5-e010d0fdfac7 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.662337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a06b8c-cc8a-4c82-9bd5-e010d0fdfac7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:21.662862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a06b8c-cc8a-4c82-9bd5-e010d0fdfac7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.664473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a06b8c-cc8a-4c82-9bd5-e010d0fdfac7 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.674948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145a661-a753-4cdb-b6f2-f9bb36ed6ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.676249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145a661-a753-4cdb-b6f2-f9bb36ed6ea0 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.676549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145a661-a753-4cdb-b6f2-f9bb36ed6ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.676817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145a661-a753-4cdb-b6f2-f9bb36ed6ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.677813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145a661-a753-4cdb-b6f2-f9bb36ed6ea0 None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=52470b620abd4cd29ab7e4752f285bb3) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:21.678117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145a661-a753-4cdb-b6f2-f9bb36ed6ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.679542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145a661-a753-4cdb-b6f2-f9bb36ed6ea0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.679943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 268/533] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:21.683384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a06b8c-cc8a-4c82-9bd5-e010d0fdfac7 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:21.691461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 266/534] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:21 2022] GET /identity/v3/role_assignments?user.id=52470b620abd4cd29ab7e4752f285bb3&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:21.691461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145a661-a753-4cdb-b6f2-f9bb36ed6ea0 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:21.801066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:21.807852 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145a661-a753-4cdb-b6f2-f9bb36ed6ea0 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.847850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe093e-6fff-4a3f-b120-ab112aadf678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.848682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe093e-6fff-4a3f-b120-ab112aadf678 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.848756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a06b8c-cc8a-4c82-9bd5-e010d0fdfac7 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:21.849116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe093e-6fff-4a3f-b120-ab112aadf678 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.849116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe093e-6fff-4a3f-b120-ab112aadf678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.849782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe093e-6fff-4a3f-b120-ab112aadf678 None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=bb5590d8961d47aa846117a236730d93) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:21.850186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe093e-6fff-4a3f-b120-ab112aadf678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.852026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe093e-6fff-4a3f-b120-ab112aadf678 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.858397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 267/535] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:21 2022] GET /identity/v3/role_assignments?user.id=bb5590d8961d47aa846117a236730d93&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:21.859571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c24e61-5c11-473b-89e6-350dae5fd6e9 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.859960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c24e61-5c11-473b-89e6-350dae5fd6e9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.860238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c24e61-5c11-473b-89e6-350dae5fd6e9 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.861865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 268/536] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:21.874164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd55b2-4ca3-422a-a8e0-49f25438b8bf None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.874508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd55b2-4ca3-422a-a8e0-49f25438b8bf None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.874716 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd55b2-4ca3-422a-a8e0-49f25438b8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.897486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-afbd55b2-4ca3-422a-a8e0-49f25438b8bf None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:21.898755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-afbd55b2-4ca3-422a-a8e0-49f25438b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:21.901071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fe2b1-aba8-486f-b9f2-07ddf2288dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:21.902192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fe2b1-aba8-486f-b9f2-07ddf2288dd2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:21.902577 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fe2b1-aba8-486f-b9f2-07ddf2288dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:21.902972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fe2b1-aba8-486f-b9f2-07ddf2288dd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:21.964973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:21.989436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fe2b1-aba8-486f-b9f2-07ddf2288dd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:21.991093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406fe2b1-aba8-486f-b9f2-07ddf2288dd2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:21.991851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 269/537] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:22.107135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 269/538] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:22.113861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd55b2-4ca3-422a-a8e0-49f25438b8bf None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:22.223633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:22.250752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd55b2-4ca3-422a-a8e0-49f25438b8bf None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:22.287064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81165ae0-e55e-4ad0-96e6-e137a189d50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:22.287701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81165ae0-e55e-4ad0-96e6-e137a189d50d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:22.287974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81165ae0-e55e-4ad0-96e6-e137a189d50d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:22.288242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81165ae0-e55e-4ad0-96e6-e137a189d50d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:22.299743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81165ae0-e55e-4ad0-96e6-e137a189d50d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:22.302230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81165ae0-e55e-4ad0-96e6-e137a189d50d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:22.316691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fe2b1-aba8-486f-b9f2-07ddf2288dd2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-81165ae0-e55e-4ad0-96e6-e137a189d50d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:22.321061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:22.322062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:22.322883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:22.322883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:22.322883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:22.322883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 270/539] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:22.329519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81165ae0-e55e-4ad0-96e6-e137a189d50d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:22.468951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:22.506735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fe2b1-aba8-486f-b9f2-07ddf2288dd2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:22.522159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81165ae0-e55e-4ad0-96e6-e137a189d50d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:22.578152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087de6a-f475-4626-8c31-1e22262f4015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:22.578789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087de6a-f475-4626-8c31-1e22262f4015 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:22.579067 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087de6a-f475-4626-8c31-1e22262f4015 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:22.579335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087de6a-f475-4626-8c31-1e22262f4015 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:22.587218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79382388-c617-462d-adf9-354966715c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:22.587826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79382388-c617-462d-adf9-354966715c2d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:22.588103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79382388-c617-462d-adf9-354966715c2d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:22.588368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79382388-c617-462d-adf9-354966715c2d None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:22.589042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79382388-c617-462d-adf9-354966715c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:22.589341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79382388-c617-462d-adf9-354966715c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:22.590725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79382388-c617-462d-adf9-354966715c2d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:22.594871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:22.602012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3087de6a-f475-4626-8c31-1e22262f4015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:22.606420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 271/540] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-3087de6a-f475-4626-8c31-1e22262f4015 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:22.607845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:22.608978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:22.609769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 270/541] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:22.618137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087de6a-f475-4626-8c31-1e22262f4015 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:22.618713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79382388-c617-462d-adf9-354966715c2d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:22.782808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79382388-c617-462d-adf9-354966715c2d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:22.796520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087de6a-f475-4626-8c31-1e22262f4015 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:22.828665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb65db-915e-4419-b2b7-35be3c78db6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:22.829663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb65db-915e-4419-b2b7-35be3c78db6b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:22.830139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb65db-915e-4419-b2b7-35be3c78db6b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:22.830632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb65db-915e-4419-b2b7-35be3c78db6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:22.838152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb65db-915e-4419-b2b7-35be3c78db6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:22.839881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb65db-915e-4419-b2b7-35be3c78db6b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:22.850197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016a146-b197-41c8-bf84-6b77ab0d095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:22.851717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016a146-b197-41c8-bf84-6b77ab0d095d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:22.852193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016a146-b197-41c8-bf84-6b77ab0d095d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:22.852684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016a146-b197-41c8-bf84-6b77ab0d095d None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:22.853548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016a146-b197-41c8-bf84-6b77ab0d095d None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:22.854045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016a146-b197-41c8-bf84-6b77ab0d095d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:22.855652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016a146-b197-41c8-bf84-6b77ab0d095d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-a4cb65db-915e-4419-b2b7-35be3c78db6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:22.861692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:22.862847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:22.863670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:22.863670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:22.863670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:22.863670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:22.863670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:22.863670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:22.863670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:22.863670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:22.873295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 272/542] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:22.877817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb65db-915e-4419-b2b7-35be3c78db6b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:22.880861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 271/543] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:22.887255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016a146-b197-41c8-bf84-6b77ab0d095d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:22.892837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:22.993047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:23.053223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016a146-b197-41c8-bf84-6b77ab0d095d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:23.092436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb65db-915e-4419-b2b7-35be3c78db6b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:23.110484 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504f817-5d14-4049-9c37-ab9df91d00a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:23.111455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504f817-5d14-4049-9c37-ab9df91d00a6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.111958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504f817-5d14-4049-9c37-ab9df91d00a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.112429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504f817-5d14-4049-9c37-ab9df91d00a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.128114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe860411-eebc-41c5-b353-0f44e378a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:23.129302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe860411-eebc-41c5-b353-0f44e378a9cf None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.129574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe860411-eebc-41c5-b353-0f44e378a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.129842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe860411-eebc-41c5-b353-0f44e378a9cf None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.130530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe860411-eebc-41c5-b353-0f44e378a9cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:23.130832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe860411-eebc-41c5-b353-0f44e378a9cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:23.132277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe860411-eebc-41c5-b353-0f44e378a9cf None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:23.134222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504f817-5d14-4049-9c37-ab9df91d00a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:23.135887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504f817-5d14-4049-9c37-ab9df91d00a6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-8504f817-5d14-4049-9c37-ab9df91d00a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:23.165435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:23.166541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:23.167596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:23.167596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:23.167596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:23.169043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 273/544] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:23.169914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 272/545] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:23.170374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4145884-c8aa-42d4-992b-ddc46dd8d28a None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.170661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4145884-c8aa-42d4-992b-ddc46dd8d28a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.170930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4145884-c8aa-42d4-992b-ddc46dd8d28a None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.172221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 274/546] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:23.178366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4145884-c8aa-42d4-992b-ddc46dd8d28a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:23.179440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504f817-5d14-4049-9c37-ab9df91d00a6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:23.253051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:23.288352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504f817-5d14-4049-9c37-ab9df91d00a6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:23.295681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4145884-c8aa-42d4-992b-ddc46dd8d28a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:23.315205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:23.331665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313c052-6006-4d47-931c-20b5b1a4ac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:23.332465 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313c052-6006-4d47-931c-20b5b1a4ac8e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.333097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313c052-6006-4d47-931c-20b5b1a4ac8e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.333542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313c052-6006-4d47-931c-20b5b1a4ac8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.334313 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135a101-f63e-4f28-813f-cda53539b6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:23.334854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313c052-6006-4d47-931c-20b5b1a4ac8e None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=cdf97e9e0340441c87316ce03d46cfac) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:23.334962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135a101-f63e-4f28-813f-cda53539b6c4 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.335192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135a101-f63e-4f28-813f-cda53539b6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.335352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313c052-6006-4d47-931c-20b5b1a4ac8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:23.335451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135a101-f63e-4f28-813f-cda53539b6c4 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.336089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135a101-f63e-4f28-813f-cda53539b6c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:23.336390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135a101-f63e-4f28-813f-cda53539b6c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:23.337321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313c052-6006-4d47-931c-20b5b1a4ac8e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:23.337879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135a101-f63e-4f28-813f-cda53539b6c4 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:23.349013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 273/547] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:23 2022] GET /identity/v3/role_assignments?user.id=cdf97e9e0340441c87316ce03d46cfac&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:23.350239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 275/548] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:23.350690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bec91-0caf-4a63-b149-5275bbe9d0a7 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.350989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bec91-0caf-4a63-b149-5275bbe9d0a7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.351264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bec91-0caf-4a63-b149-5275bbe9d0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.356058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135a101-f63e-4f28-813f-cda53539b6c4 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:23.393372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-556bec91-0caf-4a63-b149-5275bbe9d0a7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:23.394675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-556bec91-0caf-4a63-b149-5275bbe9d0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:23.466677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135a101-f63e-4f28-813f-cda53539b6c4 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:23.504352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eef098-683b-4dc9-9a62-ee1b7fe00ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:23.505132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eef098-683b-4dc9-9a62-ee1b7fe00ce9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.505414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eef098-683b-4dc9-9a62-ee1b7fe00ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.505676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eef098-683b-4dc9-9a62-ee1b7fe00ce9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.513964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eef098-683b-4dc9-9a62-ee1b7fe00ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:23.515442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eef098-683b-4dc9-9a62-ee1b7fe00ce9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-b6eef098-683b-4dc9-9a62-ee1b7fe00ce9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:23.535444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:23.537021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:23.538074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 276/549] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:23.545595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eef098-683b-4dc9-9a62-ee1b7fe00ce9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:23.546325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 274/550] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:23.589590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:23.652809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eef098-683b-4dc9-9a62-ee1b7fe00ce9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:23.689318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad33a7-b714-4159-9de3-db9983f5d74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:23.689921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad33a7-b714-4159-9de3-db9983f5d74c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.690191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad33a7-b714-4159-9de3-db9983f5d74c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.690459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad33a7-b714-4159-9de3-db9983f5d74c None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.691212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad33a7-b714-4159-9de3-db9983f5d74c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:23.691601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad33a7-b714-4159-9de3-db9983f5d74c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:23.693074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad33a7-b714-4159-9de3-db9983f5d74c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:23.712829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 277/551] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 00:17:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:23.718391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bec91-0caf-4a63-b149-5275bbe9d0a7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:23.774310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:23.843840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bec91-0caf-4a63-b149-5275bbe9d0a7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:23.894800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d279a-141e-4d01-9714-404187fa112c None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.895508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d279a-141e-4d01-9714-404187fa112c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.895508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d279a-141e-4d01-9714-404187fa112c None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.896867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af676a-04e0-419e-9dde-368b85562d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:23.897258 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 278/552] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:17:23.897486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af676a-04e0-419e-9dde-368b85562d84 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.897766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af676a-04e0-419e-9dde-368b85562d84 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.898037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af676a-04e0-419e-9dde-368b85562d84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.919291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99eba79-e9a9-4d2f-82f6-a7fb6e99e8f0 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:23.919631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99eba79-e9a9-4d2f-82f6-a7fb6e99e8f0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:23.919831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99eba79-e9a9-4d2f-82f6-a7fb6e99e8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:23.941749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-a99eba79-e9a9-4d2f-82f6-a7fb6e99e8f0 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:23.942934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a99eba79-e9a9-4d2f-82f6-a7fb6e99e8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:23.981933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af676a-04e0-419e-9dde-368b85562d84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:23.983503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af676a-04e0-419e-9dde-368b85562d84 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:24.020203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 279/553] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:24.021933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99eba79-e9a9-4d2f-82f6-a7fb6e99e8f0 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:24.036990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:24.051306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:24.138747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 275/554] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:24.146866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af676a-04e0-419e-9dde-368b85562d84 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:24.185851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99eba79-e9a9-4d2f-82f6-a7fb6e99e8f0 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:24.225058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0f4d1-9823-4517-a100-458a6525b0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:24.226042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0f4d1-9823-4517-a100-458a6525b0f2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.226447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0f4d1-9823-4517-a100-458a6525b0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.226921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0f4d1-9823-4517-a100-458a6525b0f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.244269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0f4d1-9823-4517-a100-458a6525b0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-0ce0f4d1-9823-4517-a100-458a6525b0f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:24.257107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:24.258083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:24.258904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 280/555] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:24.264541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0f4d1-9823-4517-a100-458a6525b0f2 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:24.368541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:24.380811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0f4d1-9823-4517-a100-458a6525b0f2 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:24.389525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af676a-04e0-419e-9dde-368b85562d84 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:24.422084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8db1f-7d10-4db4-b5de-0d51190adbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:24.422746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8db1f-7d10-4db4-b5de-0d51190adbeb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.423025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8db1f-7d10-4db4-b5de-0d51190adbeb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.423298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8db1f-7d10-4db4-b5de-0d51190adbeb None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.424053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8db1f-7d10-4db4-b5de-0d51190adbeb None admin] RBAC: Adding query filter params (name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:24.424440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8db1f-7d10-4db4-b5de-0d51190adbeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:24.426123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8db1f-7d10-4db4-b5de-0d51190adbeb None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:24.431760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163c185-f1cc-4fd0-bbc8-7ca657e06d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:24.432768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163c185-f1cc-4fd0-bbc8-7ca657e06d9e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.433112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163c185-f1cc-4fd0-bbc8-7ca657e06d9e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.433454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163c185-f1cc-4fd0-bbc8-7ca657e06d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.434910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 281/556] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:24.435852 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8db1f-7d10-4db4-b5de-0d51190adbeb None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:24.450589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163c185-f1cc-4fd0-bbc8-7ca657e06d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:24.452147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163c185-f1cc-4fd0-bbc8-7ca657e06d9e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:24.461336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-a163c185-f1cc-4fd0-bbc8-7ca657e06d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:24.470562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:24.471531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:24.472530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:24.472530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:24.472530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:24.473938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 276/557] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:24.475012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ee594-2d94-425f-8df6-df902dcce484 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.475292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ee594-2d94-425f-8df6-df902dcce484 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.475828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ee594-2d94-425f-8df6-df902dcce484 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.478660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 277/558] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:24.479380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ee594-2d94-425f-8df6-df902dcce484 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:24.568197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8db1f-7d10-4db4-b5de-0d51190adbeb None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:24.598046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ee594-2d94-425f-8df6-df902dcce484 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:24.608339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1a5cb-a9d7-40e9-a602-6b2c5a6e2693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:24.609004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1a5cb-a9d7-40e9-a602-6b2c5a6e2693 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.609401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1a5cb-a9d7-40e9-a602-6b2c5a6e2693 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.609748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1a5cb-a9d7-40e9-a602-6b2c5a6e2693 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.621780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1a5cb-a9d7-40e9-a602-6b2c5a6e2693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-05c1a5cb-a9d7-40e9-a602-6b2c5a6e2693 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:24.626856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:24.627832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 282/559] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:24.634203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1a5cb-a9d7-40e9-a602-6b2c5a6e2693 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:24.638214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:24.653175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d86c6-4f4b-493a-94e1-6c387a95aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:24.654879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d86c6-4f4b-493a-94e1-6c387a95aa9d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.655163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d86c6-4f4b-493a-94e1-6c387a95aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.656211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d86c6-4f4b-493a-94e1-6c387a95aa9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.673345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d86c6-4f4b-493a-94e1-6c387a95aa9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:24.675932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d86c6-4f4b-493a-94e1-6c387a95aa9d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-6f3d86c6-4f4b-493a-94e1-6c387a95aa9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:24.692550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:24.693569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:24.694449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:24.694449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:24.694449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:24.694449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 278/560] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:24.695153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc86966-cefd-44a7-80bb-bba82cced26d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.695253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc86966-cefd-44a7-80bb-bba82cced26d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.695466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc86966-cefd-44a7-80bb-bba82cced26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.717165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-afc86966-cefd-44a7-80bb-bba82cced26d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:24.718354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-afc86966-cefd-44a7-80bb-bba82cced26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:24.748911 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1a5cb-a9d7-40e9-a602-6b2c5a6e2693 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:24.771074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:24.787277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0593953-74f6-4198-a588-a51a12f35fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:24.787884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0593953-74f6-4198-a588-a51a12f35fa7 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.788154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0593953-74f6-4198-a588-a51a12f35fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.788444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0593953-74f6-4198-a588-a51a12f35fa7 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.789117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0593953-74f6-4198-a588-a51a12f35fa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:24.789564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0593953-74f6-4198-a588-a51a12f35fa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:24.790796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0593953-74f6-4198-a588-a51a12f35fa7 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:24.804524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 279/561] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:24.804524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc86966-cefd-44a7-80bb-bba82cced26d None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:24.810788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 283/562] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:24.811750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0593953-74f6-4198-a588-a51a12f35fa7 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:24.883266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:24.928038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0593953-74f6-4198-a588-a51a12f35fa7 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:24.939837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc86966-cefd-44a7-80bb-bba82cced26d None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:24.974972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cfdb6-0349-42d4-a7be-5cf9cf1f5cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:24.976375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cfdb6-0349-42d4-a7be-5cf9cf1f5cac None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.976873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cfdb6-0349-42d4-a7be-5cf9cf1f5cac None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.977323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cfdb6-0349-42d4-a7be-5cf9cf1f5cac None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.978156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cfdb6-0349-42d4-a7be-5cf9cf1f5cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:24.978634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cfdb6-0349-42d4-a7be-5cf9cf1f5cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:24.979894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f461589-3f9b-4f53-8ec5-7aab62d5a3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:24.980415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cfdb6-0349-42d4-a7be-5cf9cf1f5cac None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:24.980739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f461589-3f9b-4f53-8ec5-7aab62d5a3fa None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:24.981114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f461589-3f9b-4f53-8ec5-7aab62d5a3fa None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:24.981483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f461589-3f9b-4f53-8ec5-7aab62d5a3fa None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:24.982427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f461589-3f9b-4f53-8ec5-7aab62d5a3fa None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:24.982845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f461589-3f9b-4f53-8ec5-7aab62d5a3fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:24.984422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f461589-3f9b-4f53-8ec5-7aab62d5a3fa None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:24.994323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 280/563] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:24.995377 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f461589-3f9b-4f53-8ec5-7aab62d5a3fa None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:24.995935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 284/564] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:25.006167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cfdb6-0349-42d4-a7be-5cf9cf1f5cac None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.056083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:25.128715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cfdb6-0349-42d4-a7be-5cf9cf1f5cac None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.131671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f461589-3f9b-4f53-8ec5-7aab62d5a3fa None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.153306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:25.165914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca881b5-75d6-4297-8e6f-f2bdd8e667a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.166336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37779-ee9a-4d6f-a83c-b6966494b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.166546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca881b5-75d6-4297-8e6f-f2bdd8e667a0 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.166808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca881b5-75d6-4297-8e6f-f2bdd8e667a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.166969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37779-ee9a-4d6f-a83c-b6966494b067 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.167073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca881b5-75d6-4297-8e6f-f2bdd8e667a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.167231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37779-ee9a-4d6f-a83c-b6966494b067 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.167570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37779-ee9a-4d6f-a83c-b6966494b067 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.174419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37779-ee9a-4d6f-a83c-b6966494b067 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:25.174539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca881b5-75d6-4297-8e6f-f2bdd8e667a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:25.176420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37779-ee9a-4d6f-a83c-b6966494b067 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:25.176663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca881b5-75d6-4297-8e6f-f2bdd8e667a0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-4dc37779-ee9a-4d6f-a83c-b6966494b067 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:25.187624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:25.188569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:25.189492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:25.189492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:25.189492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:25.189492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:25.189492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:25.189492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:25.189492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:25.189492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-9ca881b5-75d6-4297-8e6f-f2bdd8e667a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:25.189923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:25.190837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:25.191779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:25.193603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 281/565] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:25.194081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 285/566] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:25.194721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37779-ee9a-4d6f-a83c-b6966494b067 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.194892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca881b5-75d6-4297-8e6f-f2bdd8e667a0 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.304846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca881b5-75d6-4297-8e6f-f2bdd8e667a0 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.311824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37779-ee9a-4d6f-a83c-b6966494b067 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.314176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:25.340904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746ee36-6e4c-4365-85a5-8e1aca2e0d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.341523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746ee36-6e4c-4365-85a5-8e1aca2e0d5e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.341799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746ee36-6e4c-4365-85a5-8e1aca2e0d5e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.342066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746ee36-6e4c-4365-85a5-8e1aca2e0d5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.349237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82097fa8-d02e-4b61-a822-e47130809377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.349868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82097fa8-d02e-4b61-a822-e47130809377 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.350148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82097fa8-d02e-4b61-a822-e47130809377 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.350673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82097fa8-d02e-4b61-a822-e47130809377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.353304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746ee36-6e4c-4365-85a5-8e1aca2e0d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-b746ee36-6e4c-4365-85a5-8e1aca2e0d5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:25.358513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:25.359615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:25.360451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 286/567] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:25.361173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746ee36-6e4c-4365-85a5-8e1aca2e0d5e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.366865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82097fa8-d02e-4b61-a822-e47130809377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:25.369591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82097fa8-d02e-4b61-a822-e47130809377 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-82097fa8-d02e-4b61-a822-e47130809377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:25.385937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:25.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:25.387760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:25.387760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:25.387760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:25.391533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 282/568] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:25.392323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82097fa8-d02e-4b61-a822-e47130809377 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.416712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:25.476620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746ee36-6e4c-4365-85a5-8e1aca2e0d5e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.502679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82097fa8-d02e-4b61-a822-e47130809377 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.511761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9c832-8a14-4517-8338-369876c31f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.512366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9c832-8a14-4517-8338-369876c31f2e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.512679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9c832-8a14-4517-8338-369876c31f2e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.512954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9c832-8a14-4517-8338-369876c31f2e None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.513653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9c832-8a14-4517-8338-369876c31f2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:25.513960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9c832-8a14-4517-8338-369876c31f2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:25.515398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9c832-8a14-4517-8338-369876c31f2e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:25.528537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 287/569] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:25.528537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9c832-8a14-4517-8338-369876c31f2e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.544662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc36f-f691-4f4a-b6ae-0a4dbede042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.545269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc36f-f691-4f4a-b6ae-0a4dbede042f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.545555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc36f-f691-4f4a-b6ae-0a4dbede042f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.545881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87efc36f-f691-4f4a-b6ae-0a4dbede042f None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.546776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efc36f-f691-4f4a-b6ae-0a4dbede042f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:25.547191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efc36f-f691-4f4a-b6ae-0a4dbede042f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:25.548769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87efc36f-f691-4f4a-b6ae-0a4dbede042f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:25.569454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 283/570] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:25.569454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc36f-f691-4f4a-b6ae-0a4dbede042f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.587560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:25.637541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9c832-8a14-4517-8338-369876c31f2e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.639857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:25.677059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa278a4-d1f2-4129-bb0b-b9602ad2674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.677659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa278a4-d1f2-4129-bb0b-b9602ad2674f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.677937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa278a4-d1f2-4129-bb0b-b9602ad2674f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.678206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa278a4-d1f2-4129-bb0b-b9602ad2674f None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.678859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa278a4-d1f2-4129-bb0b-b9602ad2674f None admin] RBAC: Adding query filter params (name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:25.679158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa278a4-d1f2-4129-bb0b-b9602ad2674f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:25.680562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa278a4-d1f2-4129-bb0b-b9602ad2674f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:25.681500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87efc36f-f691-4f4a-b6ae-0a4dbede042f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.688526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 288/571] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:25.689352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa278a4-d1f2-4129-bb0b-b9602ad2674f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.724062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e3c15-29fe-42ff-a3c8-e3dee79ce55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.724816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e3c15-29fe-42ff-a3c8-e3dee79ce55d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.725091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e3c15-29fe-42ff-a3c8-e3dee79ce55d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.726241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e3c15-29fe-42ff-a3c8-e3dee79ce55d None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.726767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e3c15-29fe-42ff-a3c8-e3dee79ce55d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:25.727018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e3c15-29fe-42ff-a3c8-e3dee79ce55d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:25.728398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e3c15-29fe-42ff-a3c8-e3dee79ce55d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:25.739930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 284/572] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:25.740763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e3c15-29fe-42ff-a3c8-e3dee79ce55d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.794378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa278a4-d1f2-4129-bb0b-b9602ad2674f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.814443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:25.828660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f781b-3ac2-4ef8-8542-da45292c2977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.829256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f781b-3ac2-4ef8-8542-da45292c2977 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.829533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f781b-3ac2-4ef8-8542-da45292c2977 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.829803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f781b-3ac2-4ef8-8542-da45292c2977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.830518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f781b-3ac2-4ef8-8542-da45292c2977 None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=43d6c2a533e849cab240257cc1450c9f) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:25.830821 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f781b-3ac2-4ef8-8542-da45292c2977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:25.833049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f781b-3ac2-4ef8-8542-da45292c2977 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:25.838320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 289/573] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/role_assignments?user.id=43d6c2a533e849cab240257cc1450c9f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:25.839091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f781b-3ac2-4ef8-8542-da45292c2977 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.839871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e3c15-29fe-42ff-a3c8-e3dee79ce55d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:25.876766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:25.885137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9b696-74d4-4dc7-89e1-4f98e1688be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:25.885795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b696-74d4-4dc7-89e1-4f98e1688be2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.886073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b696-74d4-4dc7-89e1-4f98e1688be2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.886345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b696-74d4-4dc7-89e1-4f98e1688be2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.899778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9b696-74d4-4dc7-89e1-4f98e1688be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:25.901437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9b696-74d4-4dc7-89e1-4f98e1688be2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-96c9b696-74d4-4dc7-89e1-4f98e1688be2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:25.918316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:25.919495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:25.920368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:25.920368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:25.920368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:25.920368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 285/574] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:25.920996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1ae8d-e29d-4e22-b317-820db0dc0258 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:25.920996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1ae8d-e29d-4e22-b317-820db0dc0258 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:25.921210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1ae8d-e29d-4e22-b317-820db0dc0258 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:25.926930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 286/575] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:25.926930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1ae8d-e29d-4e22-b317-820db0dc0258 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:25.963402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f781b-3ac2-4ef8-8542-da45292c2977 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.002218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa3414-e7ed-42d9-b869-9571b9a58733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.002860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa3414-e7ed-42d9-b869-9571b9a58733 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.003274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa3414-e7ed-42d9-b869-9571b9a58733 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.003615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa3414-e7ed-42d9-b869-9571b9a58733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.014674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa3414-e7ed-42d9-b869-9571b9a58733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.016226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa3414-e7ed-42d9-b869-9571b9a58733 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-f9fa3414-e7ed-42d9-b869-9571b9a58733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:26.031430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:26.032384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:26.033298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:26.033298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:26.033298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:26.033298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 290/576] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:26.034116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa3414-e7ed-42d9-b869-9571b9a58733 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.042330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1ae8d-e29d-4e22-b317-820db0dc0258 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.070587 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:26.081739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751e07f-7fc3-4a09-92b4-384a49482131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.082461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751e07f-7fc3-4a09-92b4-384a49482131 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.082739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751e07f-7fc3-4a09-92b4-384a49482131 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.083020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751e07f-7fc3-4a09-92b4-384a49482131 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.090342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751e07f-7fc3-4a09-92b4-384a49482131 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.091816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751e07f-7fc3-4a09-92b4-384a49482131 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-5751e07f-7fc3-4a09-92b4-384a49482131 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:26.104289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:26.105421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:26.106501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 287/577] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:26.110994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867786-8bb9-4404-8e20-bed43e6585b3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.111289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867786-8bb9-4404-8e20-bed43e6585b3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.111551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75867786-8bb9-4404-8e20-bed43e6585b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.126760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:26.130438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-75867786-8bb9-4404-8e20-bed43e6585b3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:26.131636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-75867786-8bb9-4404-8e20-bed43e6585b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:26.149021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa3414-e7ed-42d9-b869-9571b9a58733 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.185290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9a804-192b-400d-95f5-f878ec86068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.185895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9a804-192b-400d-95f5-f878ec86068d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.186175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9a804-192b-400d-95f5-f878ec86068d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.186450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9a804-192b-400d-95f5-f878ec86068d None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.187091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9a804-192b-400d-95f5-f878ec86068d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:26.187393 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9a804-192b-400d-95f5-f878ec86068d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.188811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9a804-192b-400d-95f5-f878ec86068d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.200659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 291/578] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:26.201474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9a804-192b-400d-95f5-f878ec86068d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.212643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 288/579] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:26.213797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75867786-8bb9-4404-8e20-bed43e6585b3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.313124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9a804-192b-400d-95f5-f878ec86068d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.348370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb5a67-5f14-4163-bd73-91858ec57f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.349484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb5a67-5f14-4163-bd73-91858ec57f9a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.349963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb5a67-5f14-4163-bd73-91858ec57f9a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.350720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb5a67-5f14-4163-bd73-91858ec57f9a None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.351614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb5a67-5f14-4163-bd73-91858ec57f9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:26.351731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75867786-8bb9-4404-8e20-bed43e6585b3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.352225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb5a67-5f14-4163-bd73-91858ec57f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.353839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb5a67-5f14-4163-bd73-91858ec57f9a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.370233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:26.376643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 292/580] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:26.377776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb5a67-5f14-4163-bd73-91858ec57f9a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.393722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07481b9-3beb-4cfa-b7bc-3180e00bd0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.394649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07481b9-3beb-4cfa-b7bc-3180e00bd0ff None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.395125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07481b9-3beb-4cfa-b7bc-3180e00bd0ff None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.395652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07481b9-3beb-4cfa-b7bc-3180e00bd0ff None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.396670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07481b9-3beb-4cfa-b7bc-3180e00bd0ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:26.397241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07481b9-3beb-4cfa-b7bc-3180e00bd0ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.398927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07481b9-3beb-4cfa-b7bc-3180e00bd0ff None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.412847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:26.422704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 289/581] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:26.423927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07481b9-3beb-4cfa-b7bc-3180e00bd0ff None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.499951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb5a67-5f14-4163-bd73-91858ec57f9a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.538271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4f35e-436b-41a2-ac0f-f4422921ce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.538932 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4f35e-436b-41a2-ac0f-f4422921ce0c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.539502 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4f35e-436b-41a2-ac0f-f4422921ce0c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.539784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4f35e-436b-41a2-ac0f-f4422921ce0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.542164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07481b9-3beb-4cfa-b7bc-3180e00bd0ff None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.546627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4f35e-436b-41a2-ac0f-f4422921ce0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.548079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4f35e-436b-41a2-ac0f-f4422921ce0c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-f7f4f35e-436b-41a2-ac0f-f4422921ce0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:26.559876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:26.560798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:26.561681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:26.561681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:26.561681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:26.561681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:26.561681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:26.561681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:26.561681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:26.561681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:26.566426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 293/582] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:26.567137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4f35e-436b-41a2-ac0f-f4422921ce0c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.581104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba694693-9e51-4240-89bb-9e3a93db0df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.581804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba694693-9e51-4240-89bb-9e3a93db0df6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.582098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba694693-9e51-4240-89bb-9e3a93db0df6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.582351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba694693-9e51-4240-89bb-9e3a93db0df6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.590065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba694693-9e51-4240-89bb-9e3a93db0df6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.591497 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba694693-9e51-4240-89bb-9e3a93db0df6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-ba694693-9e51-4240-89bb-9e3a93db0df6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:26.603788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:26.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 290/583] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:26.605734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba694693-9e51-4240-89bb-9e3a93db0df6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.637157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:26.642930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:26.683080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4f35e-436b-41a2-ac0f-f4422921ce0c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.712332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba694693-9e51-4240-89bb-9e3a93db0df6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.720194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6be7b-7d8e-4309-b481-9454284ee16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.720965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6be7b-7d8e-4309-b481-9454284ee16d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.721310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6be7b-7d8e-4309-b481-9454284ee16d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.721648 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6be7b-7d8e-4309-b481-9454284ee16d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.734179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6be7b-7d8e-4309-b481-9454284ee16d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.735757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6be7b-7d8e-4309-b481-9454284ee16d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.747973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447370c-631b-49b4-8834-636a41502342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.748694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447370c-631b-49b4-8834-636a41502342 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.749034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447370c-631b-49b4-8834-636a41502342 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.749367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447370c-631b-49b4-8834-636a41502342 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-27b6be7b-7d8e-4309-b481-9454284ee16d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:26.751243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:26.752330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:26.753509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:26.753509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:26.753509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:26.756644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 294/584] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:26.757920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6be7b-7d8e-4309-b481-9454284ee16d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.763048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447370c-631b-49b4-8834-636a41502342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-4447370c-631b-49b4-8834-636a41502342 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:26.767865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:26.769335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:26.770194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 291/585] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:26.770865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447370c-631b-49b4-8834-636a41502342 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.870101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6be7b-7d8e-4309-b481-9454284ee16d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.874548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:26.881878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447370c-631b-49b4-8834-636a41502342 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:26.884729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:26.907615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fa222-db55-4f70-8839-33dcea8c02f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.908241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fa222-db55-4f70-8839-33dcea8c02f4 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.908544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fa222-db55-4f70-8839-33dcea8c02f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.908823 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fa222-db55-4f70-8839-33dcea8c02f4 None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.909787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fa222-db55-4f70-8839-33dcea8c02f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:26.910102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fa222-db55-4f70-8839-33dcea8c02f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.911621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fa222-db55-4f70-8839-33dcea8c02f4 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.922681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733d055-1f92-4e79-ad0b-17c59674c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:26.923287 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733d055-1f92-4e79-ad0b-17c59674c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:26.923561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733d055-1f92-4e79-ad0b-17c59674c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:26.923827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733d055-1f92-4e79-ad0b-17c59674c1c4 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:26.924563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 295/586] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:26.925752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fa222-db55-4f70-8839-33dcea8c02f4 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:26.929868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733d055-1f92-4e79-ad0b-17c59674c1c4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:26.930162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733d055-1f92-4e79-ad0b-17c59674c1c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:26.931610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b733d055-1f92-4e79-ad0b-17c59674c1c4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:26.950105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 292/587] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:26.953371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733d055-1f92-4e79-ad0b-17c59674c1c4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.031273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fa222-db55-4f70-8839-33dcea8c02f4 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.061105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b733d055-1f92-4e79-ad0b-17c59674c1c4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.065572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ca57e-22f6-4922-ad31-45f8df510467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.066587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ca57e-22f6-4922-ad31-45f8df510467 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.066867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ca57e-22f6-4922-ad31-45f8df510467 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.067229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ca57e-22f6-4922-ad31-45f8df510467 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.067889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ca57e-22f6-4922-ad31-45f8df510467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:27.068186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ca57e-22f6-4922-ad31-45f8df510467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.069687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ca57e-22f6-4922-ad31-45f8df510467 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.086324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 296/588] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:27.086324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ca57e-22f6-4922-ad31-45f8df510467 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.102394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405921f-0520-49ed-b88f-ca6bf6e525b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.102996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405921f-0520-49ed-b88f-ca6bf6e525b6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.103278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405921f-0520-49ed-b88f-ca6bf6e525b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.103531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405921f-0520-49ed-b88f-ca6bf6e525b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.104184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405921f-0520-49ed-b88f-ca6bf6e525b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:27.104538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405921f-0520-49ed-b88f-ca6bf6e525b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.105860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405921f-0520-49ed-b88f-ca6bf6e525b6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.114102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 293/589] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:27.114998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405921f-0520-49ed-b88f-ca6bf6e525b6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.124353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:27.131707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:27.194091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ca57e-22f6-4922-ad31-45f8df510467 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.221559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405921f-0520-49ed-b88f-ca6bf6e525b6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.231426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc852fa0-bec1-43d6-9f4d-461b43227d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.232096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc852fa0-bec1-43d6-9f4d-461b43227d92 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.232368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc852fa0-bec1-43d6-9f4d-461b43227d92 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.232662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc852fa0-bec1-43d6-9f4d-461b43227d92 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/fb204040ab2f4168b35c3e78bacb849f/roles/87ec4f3044a84c9b8459c5458278b04c` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.248003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc852fa0-bec1-43d6-9f4d-461b43227d92 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=fb204040ab2f4168b35c3e78bacb849f, role_id=87ec4f3044a84c9b8459c5458278b04c)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.249494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc852fa0-bec1-43d6-9f4d-461b43227d92 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.272528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 297/590] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:17:27 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/fb204040ab2f4168b35c3e78bacb849f/roles/87ec4f3044a84c9b8459c5458278b04c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:27.272528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc852fa0-bec1-43d6-9f4d-461b43227d92 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.275488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091d525-539a-49f1-a343-fd40203ad3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.276887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091d525-539a-49f1-a343-fd40203ad3c0 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.277281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091d525-539a-49f1-a343-fd40203ad3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.278054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091d525-539a-49f1-a343-fd40203ad3c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.296469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091d525-539a-49f1-a343-fd40203ad3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.299144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091d525-539a-49f1-a343-fd40203ad3c0 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f091d525-539a-49f1-a343-fd40203ad3c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:27.322836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:27.324006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:27.325052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:27.325052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:27.325052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:27.326361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 294/591] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:27.327248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091d525-539a-49f1-a343-fd40203ad3c0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.379752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:27.400941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc852fa0-bec1-43d6-9f4d-461b43227d92 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.419314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:27.443092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f303e3f-141e-418d-aa66-6bcb17ba7f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.444137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f303e3f-141e-418d-aa66-6bcb17ba7f08 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.444609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f303e3f-141e-418d-aa66-6bcb17ba7f08 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.445128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f303e3f-141e-418d-aa66-6bcb17ba7f08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.452830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f303e3f-141e-418d-aa66-6bcb17ba7f08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.454297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f303e3f-141e-418d-aa66-6bcb17ba7f08 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-9f303e3f-141e-418d-aa66-6bcb17ba7f08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:27.466851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:27.467788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:27.468667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:27.468667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:27.468667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:27.468667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:27.468667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:27.468667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:27.468667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:27.468667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:27.468667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 298/592] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:27.469143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f303e3f-141e-418d-aa66-6bcb17ba7f08 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.470347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091d525-539a-49f1-a343-fd40203ad3c0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.506949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada8b27-7fde-4c03-a3f9-2b13fc6fb894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.507972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada8b27-7fde-4c03-a3f9-2b13fc6fb894 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.508557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada8b27-7fde-4c03-a3f9-2b13fc6fb894 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.509021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada8b27-7fde-4c03-a3f9-2b13fc6fb894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.521216 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada8b27-7fde-4c03-a3f9-2b13fc6fb894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.523053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada8b27-7fde-4c03-a3f9-2b13fc6fb894 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-dada8b27-7fde-4c03-a3f9-2b13fc6fb894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:27.538320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:27.539444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:27.540322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:27.540322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:27.540322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:27.542435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 295/593] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:27.543433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada8b27-7fde-4c03-a3f9-2b13fc6fb894 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.582098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f303e3f-141e-418d-aa66-6bcb17ba7f08 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.619343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75715ad1-fd47-478c-b568-4836d6ecb34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.619951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75715ad1-fd47-478c-b568-4836d6ecb34b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.620232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75715ad1-fd47-478c-b568-4836d6ecb34b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.620519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75715ad1-fd47-478c-b568-4836d6ecb34b None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.621169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75715ad1-fd47-478c-b568-4836d6ecb34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:27.621470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75715ad1-fd47-478c-b568-4836d6ecb34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.622854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75715ad1-fd47-478c-b568-4836d6ecb34b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.640443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 299/594] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:27.640443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75715ad1-fd47-478c-b568-4836d6ecb34b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.640907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:27.658470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada8b27-7fde-4c03-a3f9-2b13fc6fb894 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.679051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:27.698659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d81a8-49ef-47ed-a1e2-0da53456cb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.699596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d81a8-49ef-47ed-a1e2-0da53456cb7d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.700112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d81a8-49ef-47ed-a1e2-0da53456cb7d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.700606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d81a8-49ef-47ed-a1e2-0da53456cb7d None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.701487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d81a8-49ef-47ed-a1e2-0da53456cb7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:27.702134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d81a8-49ef-47ed-a1e2-0da53456cb7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.703825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d81a8-49ef-47ed-a1e2-0da53456cb7d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.717615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 296/595] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:27.718760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d81a8-49ef-47ed-a1e2-0da53456cb7d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.747801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75715ad1-fd47-478c-b568-4836d6ecb34b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.786899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237d65a-e5f9-4f6c-aa82-94b3c489fd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.787511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237d65a-e5f9-4f6c-aa82-94b3c489fd58 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.787784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237d65a-e5f9-4f6c-aa82-94b3c489fd58 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.788054 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237d65a-e5f9-4f6c-aa82-94b3c489fd58 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.788773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237d65a-e5f9-4f6c-aa82-94b3c489fd58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:27.789030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237d65a-e5f9-4f6c-aa82-94b3c489fd58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.790408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237d65a-e5f9-4f6c-aa82-94b3c489fd58 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.807884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 300/596] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:27.807884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237d65a-e5f9-4f6c-aa82-94b3c489fd58 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.832758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d81a8-49ef-47ed-a1e2-0da53456cb7d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.869012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189ce74-fe17-4745-9489-5c59a45cc2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.869722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189ce74-fe17-4745-9489-5c59a45cc2c9 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.870061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189ce74-fe17-4745-9489-5c59a45cc2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.870397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189ce74-fe17-4745-9489-5c59a45cc2c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.872732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:27.877791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189ce74-fe17-4745-9489-5c59a45cc2c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.880213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189ce74-fe17-4745-9489-5c59a45cc2c9 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-e189ce74-fe17-4745-9489-5c59a45cc2c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:27.891611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:27.892649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:27.893873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:27.893873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:27.893873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:27.893873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:27.893873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:27.893873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:27.893873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:27.893873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:27.894821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 297/597] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:27.896956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96933767-ed3f-458d-af44-7395a491f37e None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.897387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96933767-ed3f-458d-af44-7395a491f37e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.897807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96933767-ed3f-458d-af44-7395a491f37e None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.899588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 298/598] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:27.900338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96933767-ed3f-458d-af44-7395a491f37e None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:27.919669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237d65a-e5f9-4f6c-aa82-94b3c489fd58 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:27.943616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:27.957616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d32cc-0027-431a-9298-8d0ba666b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:27.958280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d32cc-0027-431a-9298-8d0ba666b216 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:27.958557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d32cc-0027-431a-9298-8d0ba666b216 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:27.958831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d32cc-0027-431a-9298-8d0ba666b216 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/52470b620abd4cd29ab7e4752f285bb3/roles/87ec4f3044a84c9b8459c5458278b04c` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:27.974763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d32cc-0027-431a-9298-8d0ba666b216 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=52470b620abd4cd29ab7e4752f285bb3, role_id=87ec4f3044a84c9b8459c5458278b04c)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:27.976620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d32cc-0027-431a-9298-8d0ba666b216 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:27.990966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 301/599] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:17:27 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/52470b620abd4cd29ab7e4752f285bb3/roles/87ec4f3044a84c9b8459c5458278b04c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:27.993302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d32cc-0027-431a-9298-8d0ba666b216 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:28.018152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96933767-ed3f-458d-af44-7395a491f37e None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:28.069072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3bd84-6d3c-4058-bbc1-682edfd12fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:28.070539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3bd84-6d3c-4058-bbc1-682edfd12fd1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:28.070966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3bd84-6d3c-4058-bbc1-682edfd12fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:28.071415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3bd84-6d3c-4058-bbc1-682edfd12fd1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:28.080482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3bd84-6d3c-4058-bbc1-682edfd12fd1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:28.082540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3bd84-6d3c-4058-bbc1-682edfd12fd1 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-47a3bd84-6d3c-4058-bbc1-682edfd12fd1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:28.097318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:28.098783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 00:17:28.099957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:28.107419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 299/600] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:28.108923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d87a9-4fe0-49d9-bfee-5131dcfe0fdd None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:28.109212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d87a9-4fe0-49d9-bfee-5131dcfe0fdd None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:28.109491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d87a9-4fe0-49d9-bfee-5131dcfe0fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:28.132703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-070d87a9-4fe0-49d9-bfee-5131dcfe0fdd None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:28.133915 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-070d87a9-4fe0-49d9-bfee-5131dcfe0fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:28.136377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d32cc-0027-431a-9298-8d0ba666b216 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:28.159132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:28.191205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64c9a9-341f-4eed-b5a8-99c5931be496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:28.191892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64c9a9-341f-4eed-b5a8-99c5931be496 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:28.192141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64c9a9-341f-4eed-b5a8-99c5931be496 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:28.192420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64c9a9-341f-4eed-b5a8-99c5931be496 None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:28.193179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64c9a9-341f-4eed-b5a8-99c5931be496 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:28.193458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64c9a9-341f-4eed-b5a8-99c5931be496 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:28.194960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64c9a9-341f-4eed-b5a8-99c5931be496 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:28.208691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 302/601] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:28.209547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64c9a9-341f-4eed-b5a8-99c5931be496 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:28.219780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 300/602] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:28.220669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d87a9-4fe0-49d9-bfee-5131dcfe0fdd None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:28.257101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:28.340630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d87a9-4fe0-49d9-bfee-5131dcfe0fdd None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:28.347088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64c9a9-341f-4eed-b5a8-99c5931be496 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:28.385487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a138aa3-70ea-4295-998d-3190c57d775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:28.386388 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a138aa3-70ea-4295-998d-3190c57d775a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:28.386813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a138aa3-70ea-4295-998d-3190c57d775a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:28.387256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a138aa3-70ea-4295-998d-3190c57d775a None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:28.388124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a138aa3-70ea-4295-998d-3190c57d775a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:28.388601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a138aa3-70ea-4295-998d-3190c57d775a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:28.390212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a138aa3-70ea-4295-998d-3190c57d775a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:28.398981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07ef17-c045-43bf-b3a2-cadb5eddf6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:28.399662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07ef17-c045-43bf-b3a2-cadb5eddf6f0 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:28.399955 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07ef17-c045-43bf-b3a2-cadb5eddf6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:28.400212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07ef17-c045-43bf-b3a2-cadb5eddf6f0 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/bb5590d8961d47aa846117a236730d93/roles/87ec4f3044a84c9b8459c5458278b04c` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:28.426413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07ef17-c045-43bf-b3a2-cadb5eddf6f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=bb5590d8961d47aa846117a236730d93, role_id=87ec4f3044a84c9b8459c5458278b04c)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:28.427797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07ef17-c045-43bf-b3a2-cadb5eddf6f0 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:28.430110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 303/603] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 1 00:17:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:28.431027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a138aa3-70ea-4295-998d-3190c57d775a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:28.456998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 301/604] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:17:28 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/bb5590d8961d47aa846117a236730d93/roles/87ec4f3044a84c9b8459c5458278b04c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:28.474368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:28.563548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a138aa3-70ea-4295-998d-3190c57d775a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:28.615138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3c91e-e541-4079-a36f-22874f1d9254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:28.616082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3c91e-e541-4079-a36f-22874f1d9254 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:28.616527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3c91e-e541-4079-a36f-22874f1d9254 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:28.616946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3c91e-e541-4079-a36f-22874f1d9254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:28.630564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07ef17-c045-43bf-b3a2-cadb5eddf6f0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:28.632758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3c91e-e541-4079-a36f-22874f1d9254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:28.634582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f3c91e-e541-4079-a36f-22874f1d9254 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-19f3c91e-e541-4079-a36f-22874f1d9254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:28.652025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:28.653252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:28.654325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:28.654325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:28.654325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:28.656187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 304/605] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:28.664888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3c91e-e541-4079-a36f-22874f1d9254 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:28.742279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:28.769908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:28.778829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07ef17-c045-43bf-b3a2-cadb5eddf6f0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:28.789124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3c91e-e541-4079-a36f-22874f1d9254 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:28.814099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146db7a-5ad1-44c6-9eea-97727e1999c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:28.814731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146db7a-5ad1-44c6-9eea-97727e1999c3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:28.815006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146db7a-5ad1-44c6-9eea-97727e1999c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:28.815275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146db7a-5ad1-44c6-9eea-97727e1999c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:28.826471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146db7a-5ad1-44c6-9eea-97727e1999c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:28.829107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713a8af-623e-4f1d-abda-ff92798bc977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:28.829939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713a8af-623e-4f1d-abda-ff92798bc977 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:28.830457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713a8af-623e-4f1d-abda-ff92798bc977 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:28.830942 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713a8af-623e-4f1d-abda-ff92798bc977 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-3146db7a-5ad1-44c6-9eea-97727e1999c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:28.831442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:28.832313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:28.837303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 302/606] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:28.837982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713a8af-623e-4f1d-abda-ff92798bc977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:28.837982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713a8af-623e-4f1d-abda-ff92798bc977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:28.839158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713a8af-623e-4f1d-abda-ff92798bc977 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:28.841294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146db7a-5ad1-44c6-9eea-97727e1999c3 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:28.861575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 305/607] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:28.868347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713a8af-623e-4f1d-abda-ff92798bc977 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:28.965210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146db7a-5ad1-44c6-9eea-97727e1999c3 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:29.029953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efd47f-cf1f-4999-90d2-3b5551cb0f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:29.030479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efd47f-cf1f-4999-90d2-3b5551cb0f76 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:29.030753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efd47f-cf1f-4999-90d2-3b5551cb0f76 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:29.031023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efd47f-cf1f-4999-90d2-3b5551cb0f76 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:29.031685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efd47f-cf1f-4999-90d2-3b5551cb0f76 None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:29.031987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efd47f-cf1f-4999-90d2-3b5551cb0f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:29.034769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efd47f-cf1f-4999-90d2-3b5551cb0f76 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:29.042903 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:29.051086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 303/608] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:29.054754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713a8af-623e-4f1d-abda-ff92798bc977 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:29.057079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efd47f-cf1f-4999-90d2-3b5551cb0f76 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:29.113742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcab07-1de5-4979-9188-f8e4cdeeca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:29.114531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcab07-1de5-4979-9188-f8e4cdeeca48 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:29.115139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcab07-1de5-4979-9188-f8e4cdeeca48 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:29.115417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcab07-1de5-4979-9188-f8e4cdeeca48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:29.123386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcab07-1de5-4979-9188-f8e4cdeeca48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:29.124884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcab07-1de5-4979-9188-f8e4cdeeca48 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-9fbcab07-1de5-4979-9188-f8e4cdeeca48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:29.138016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:29.139646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:29.140804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:29.140804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:29.140804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:29.140804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:29.140804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:29.140804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:29.140804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:29.140804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:29.140804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 306/609] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:29.145761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcab07-1de5-4979-9188-f8e4cdeeca48 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:29.160202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:29.265217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcab07-1de5-4979-9188-f8e4cdeeca48 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:29.305568 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6599af5-6a01-442e-9370-636d5f74a827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:29.306187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6599af5-6a01-442e-9370-636d5f74a827 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:29.306465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6599af5-6a01-442e-9370-636d5f74a827 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:29.306994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6599af5-6a01-442e-9370-636d5f74a827 None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:29.307687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6599af5-6a01-442e-9370-636d5f74a827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:29.307989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6599af5-6a01-442e-9370-636d5f74a827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:29.309452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6599af5-6a01-442e-9370-636d5f74a827 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:29.318656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efd47f-cf1f-4999-90d2-3b5551cb0f76 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:29.323498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 307/610] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:29.329839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6599af5-6a01-442e-9370-636d5f74a827 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:29.391074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad613532-14a3-4762-88ac-7cc1cc5e873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:29.391827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad613532-14a3-4762-88ac-7cc1cc5e873d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:29.392104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad613532-14a3-4762-88ac-7cc1cc5e873d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:29.392370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad613532-14a3-4762-88ac-7cc1cc5e873d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:29.408090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad613532-14a3-4762-88ac-7cc1cc5e873d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:29.410236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad613532-14a3-4762-88ac-7cc1cc5e873d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:29.415864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-ad613532-14a3-4762-88ac-7cc1cc5e873d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:29.428618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:29.429763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:29.430990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:29.430990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:29.430990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:29.430990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 304/611] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:29.435550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad613532-14a3-4762-88ac-7cc1cc5e873d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:29.484524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6599af5-6a01-442e-9370-636d5f74a827 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:29.491977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:29.551882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad613532-14a3-4762-88ac-7cc1cc5e873d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:29.552620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ab11c-b45e-4055-8dd1-0b5a57efaf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:29.553387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ab11c-b45e-4055-8dd1-0b5a57efaf65 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:29.553763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ab11c-b45e-4055-8dd1-0b5a57efaf65 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:29.554387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ab11c-b45e-4055-8dd1-0b5a57efaf65 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/cdf97e9e0340441c87316ce03d46cfac/roles/87ec4f3044a84c9b8459c5458278b04c` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:29.574832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ab11c-b45e-4055-8dd1-0b5a57efaf65 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=cdf97e9e0340441c87316ce03d46cfac, role_id=87ec4f3044a84c9b8459c5458278b04c)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:29.576349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ab11c-b45e-4055-8dd1-0b5a57efaf65 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:29.592544 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c441a-7679-432d-ab66-f4f21c14caa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:29.593257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 308/612] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:17:29 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/cdf97e9e0340441c87316ce03d46cfac/roles/87ec4f3044a84c9b8459c5458278b04c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:29.593583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c441a-7679-432d-ab66-f4f21c14caa4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:29.594005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c441a-7679-432d-ab66-f4f21c14caa4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:29.594406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c441a-7679-432d-ab66-f4f21c14caa4 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:29.595289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c441a-7679-432d-ab66-f4f21c14caa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:29.595720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c441a-7679-432d-ab66-f4f21c14caa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:29.597857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c441a-7679-432d-ab66-f4f21c14caa4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:29.602017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70920a-e9e6-4b19-84bb-0993256b4163 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:29.602400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70920a-e9e6-4b19-84bb-0993256b4163 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:29.602765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70920a-e9e6-4b19-84bb-0993256b4163 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:29.608403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 309/613] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:29.613136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508b652-81b2-4ee8-856d-92724648fd6b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:29.613558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508b652-81b2-4ee8-856d-92724648fd6b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:29.614926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508b652-81b2-4ee8-856d-92724648fd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:29.618148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 305/614] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:29.629829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c441a-7679-432d-ab66-f4f21c14caa4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:29.648208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2508b652-81b2-4ee8-856d-92724648fd6b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:29.650389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2508b652-81b2-4ee8-856d-92724648fd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:29.740298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 310/615] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:29.818001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c441a-7679-432d-ab66-f4f21c14caa4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:29.835839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:29.863112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628af9-526e-416d-a593-68f2ba7dda4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:29.863880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628af9-526e-416d-a593-68f2ba7dda4c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:29.864165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628af9-526e-416d-a593-68f2ba7dda4c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:29.864440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628af9-526e-416d-a593-68f2ba7dda4c None admin] PATH_INFO: `/v3/users/swift` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:29.871966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628af9-526e-416d-a593-68f2ba7dda4c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:29.873504 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628af9-526e-416d-a593-68f2ba7dda4c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-57628af9-526e-416d-a593-68f2ba7dda4c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:29.885558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:29.886486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:29.887396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 306/616] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:29.894768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508b652-81b2-4ee8-856d-92724648fd6b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:29.947555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:30.013580 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508b652-81b2-4ee8-856d-92724648fd6b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:30.059838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ca02-254e-407c-a3bd-00a90adc0368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:30.060865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ca02-254e-407c-a3bd-00a90adc0368 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.061372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ca02-254e-407c-a3bd-00a90adc0368 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.061844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ca02-254e-407c-a3bd-00a90adc0368 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.062824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ca02-254e-407c-a3bd-00a90adc0368 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:30.063330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ca02-254e-407c-a3bd-00a90adc0368 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:30.065052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ca02-254e-407c-a3bd-00a90adc0368 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:30.087900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 311/617] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 00:17:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:30.090685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628af9-526e-416d-a593-68f2ba7dda4c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:30.093976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ca02-254e-407c-a3bd-00a90adc0368 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:30.222535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628af9-526e-416d-a593-68f2ba7dda4c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:30.232315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ca02-254e-407c-a3bd-00a90adc0368 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:30.233995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:30.278511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75815ff-c1c9-4635-b708-867f110c7ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:30.279208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75815ff-c1c9-4635-b708-867f110c7ce0 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.279566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75815ff-c1c9-4635-b708-867f110c7ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.279844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75815ff-c1c9-4635-b708-867f110c7ce0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.291602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75815ff-c1c9-4635-b708-867f110c7ce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:30.291947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1a3c2-5458-47c5-8c5e-6c38d277ab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:30.292616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1a3c2-5458-47c5-8c5e-6c38d277ab70 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.292891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1a3c2-5458-47c5-8c5e-6c38d277ab70 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.293158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1a3c2-5458-47c5-8c5e-6c38d277ab70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-c75815ff-c1c9-4635-b708-867f110c7ce0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:30.297154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:30.298051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:30.298874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 307/618] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:30.309371 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1a3c2-5458-47c5-8c5e-6c38d277ab70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:30.309828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75815ff-c1c9-4635-b708-867f110c7ce0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:30.310972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1a3c2-5458-47c5-8c5e-6c38d277ab70 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-27b1a3c2-5458-47c5-8c5e-6c38d277ab70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:30.333277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:30.334411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:30.335280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:30.335280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:30.335280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:30.335280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 312/619] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:30.345537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1a3c2-5458-47c5-8c5e-6c38d277ab70 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:30.366983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:30.464522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1a3c2-5458-47c5-8c5e-6c38d277ab70 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:30.498214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07e8be-f02f-40c2-b3ba-eb7273e79a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:30.499304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07e8be-f02f-40c2-b3ba-eb7273e79a9c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.499935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07e8be-f02f-40c2-b3ba-eb7273e79a9c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.500318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07e8be-f02f-40c2-b3ba-eb7273e79a9c None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.501136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07e8be-f02f-40c2-b3ba-eb7273e79a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:30.501523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07e8be-f02f-40c2-b3ba-eb7273e79a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:30.503321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07e8be-f02f-40c2-b3ba-eb7273e79a9c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:30.506902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75815ff-c1c9-4635-b708-867f110c7ce0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:30.517629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 313/620] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:30.519833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331b32b-c397-4105-af7f-5f6e58d3e5a7 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.520241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331b32b-c397-4105-af7f-5f6e58d3e5a7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.520647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331b32b-c397-4105-af7f-5f6e58d3e5a7 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.522246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 314/621] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:30.527131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331b32b-c397-4105-af7f-5f6e58d3e5a7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:30.544345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:30.550276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67603aa-0e95-4167-b4b3-e55ecf1c536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:30.551386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67603aa-0e95-4167-b4b3-e55ecf1c536a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.551817 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67603aa-0e95-4167-b4b3-e55ecf1c536a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.552264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67603aa-0e95-4167-b4b3-e55ecf1c536a None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.553125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67603aa-0e95-4167-b4b3-e55ecf1c536a None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:30.553574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67603aa-0e95-4167-b4b3-e55ecf1c536a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:30.555538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67603aa-0e95-4167-b4b3-e55ecf1c536a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:30.571321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 308/622] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:30.573725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e2c1-e0b7-48ec-8468-d1cdeebd047f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.574166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e2c1-e0b7-48ec-8468-d1cdeebd047f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.574589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e2c1-e0b7-48ec-8468-d1cdeebd047f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.597815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-84a2e2c1-e0b7-48ec-8468-d1cdeebd047f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:30.599904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-84a2e2c1-e0b7-48ec-8468-d1cdeebd047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:30.648754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331b32b-c397-4105-af7f-5f6e58d3e5a7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:30.679012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 309/623] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:30.679885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e2c1-e0b7-48ec-8468-d1cdeebd047f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:30.698947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09af13-0d18-497c-9341-e1a5824c3cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:30.699958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09af13-0d18-497c-9341-e1a5824c3cde None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.700477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09af13-0d18-497c-9341-e1a5824c3cde None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.701515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09af13-0d18-497c-9341-e1a5824c3cde None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.704902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:30.714184 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09af13-0d18-497c-9341-e1a5824c3cde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:30.716147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09af13-0d18-497c-9341-e1a5824c3cde None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-ba09af13-0d18-497c-9341-e1a5824c3cde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:30.732046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:30.732986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:30.733888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:30.733888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:30.733888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:30.733888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:30.733888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:30.733888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:30.733888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:30.733888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:30.736000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 315/624] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:30.737974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df919fb4-a612-43b2-b03e-afd83dad1d04 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.738443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df919fb4-a612-43b2-b03e-afd83dad1d04 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.738921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df919fb4-a612-43b2-b03e-afd83dad1d04 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.744517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 316/625] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:30.745517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df919fb4-a612-43b2-b03e-afd83dad1d04 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:30.789219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e2c1-e0b7-48ec-8468-d1cdeebd047f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:30.824982 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fecab3-795f-42d1-934d-b1db79040649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:30.825613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fecab3-795f-42d1-934d-b1db79040649 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.825892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fecab3-795f-42d1-934d-b1db79040649 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.826159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fecab3-795f-42d1-934d-b1db79040649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.827553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:30.837872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fecab3-795f-42d1-934d-b1db79040649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:30.839374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fecab3-795f-42d1-934d-b1db79040649 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-59fecab3-795f-42d1-934d-b1db79040649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:30.855379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:30.856338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:30.857262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:30.857262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:30.857262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:30.860266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df919fb4-a612-43b2-b03e-afd83dad1d04 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:30.861261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 310/626] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:30.862619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85990d-15d2-418f-a7ec-0b2ea9f63b37 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.863028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85990d-15d2-418f-a7ec-0b2ea9f63b37 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.863230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85990d-15d2-418f-a7ec-0b2ea9f63b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.882412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3e85990d-15d2-418f-a7ec-0b2ea9f63b37 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:30.883605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3e85990d-15d2-418f-a7ec-0b2ea9f63b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:30.900910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6e468-fecf-4dba-b30b-ad7538958eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:30.902116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6e468-fecf-4dba-b30b-ad7538958eef None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:30.902999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6e468-fecf-4dba-b30b-ad7538958eef None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:30.903484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6e468-fecf-4dba-b30b-ad7538958eef None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:30.904385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6e468-fecf-4dba-b30b-ad7538958eef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:30.904923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6e468-fecf-4dba-b30b-ad7538958eef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:30.907389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b6e468-fecf-4dba-b30b-ad7538958eef None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:30.924539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 317/627] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:30.926014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6e468-fecf-4dba-b30b-ad7538958eef None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:30.961548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 311/628] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:30.962633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85990d-15d2-418f-a7ec-0b2ea9f63b37 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:30.972873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:31.047863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6e468-fecf-4dba-b30b-ad7538958eef None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.085193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad906c3-8d82-47ce-8047-d5c10284a252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.085901 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad906c3-8d82-47ce-8047-d5c10284a252 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.086178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad906c3-8d82-47ce-8047-d5c10284a252 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.086444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad906c3-8d82-47ce-8047-d5c10284a252 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.087081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad906c3-8d82-47ce-8047-d5c10284a252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:31.087379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad906c3-8d82-47ce-8047-d5c10284a252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:31.089046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad906c3-8d82-47ce-8047-d5c10284a252 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:31.090631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:31.092734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85990d-15d2-418f-a7ec-0b2ea9f63b37 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.103457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 318/629] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:31.104429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad906c3-8d82-47ce-8047-d5c10284a252 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.131837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088025ea-5b4f-4b46-a12e-5db8186a75ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.132573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088025ea-5b4f-4b46-a12e-5db8186a75ac None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.132855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088025ea-5b4f-4b46-a12e-5db8186a75ac None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.133233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088025ea-5b4f-4b46-a12e-5db8186a75ac None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/43d6c2a533e849cab240257cc1450c9f/roles/87ec4f3044a84c9b8459c5458278b04c` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.151291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088025ea-5b4f-4b46-a12e-5db8186a75ac None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=43d6c2a533e849cab240257cc1450c9f, role_id=87ec4f3044a84c9b8459c5458278b04c)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:31.152873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088025ea-5b4f-4b46-a12e-5db8186a75ac None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:31.169780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 312/630] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:17:30 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/43d6c2a533e849cab240257cc1450c9f/roles/87ec4f3044a84c9b8459c5458278b04c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:31.171005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088025ea-5b4f-4b46-a12e-5db8186a75ac None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.256838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:31.277525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad906c3-8d82-47ce-8047-d5c10284a252 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.310077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088025ea-5b4f-4b46-a12e-5db8186a75ac None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.329407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fea964-845a-49f6-bc34-db9a439dee7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.330672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fea964-845a-49f6-bc34-db9a439dee7e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.331799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fea964-845a-49f6-bc34-db9a439dee7e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.332201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fea964-845a-49f6-bc34-db9a439dee7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.347953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fea964-845a-49f6-bc34-db9a439dee7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:31.351418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20490df-21cf-4075-a5de-9c95896b5778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.352233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20490df-21cf-4075-a5de-9c95896b5778 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.352680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20490df-21cf-4075-a5de-9c95896b5778 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.353093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20490df-21cf-4075-a5de-9c95896b5778 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-49fea964-845a-49f6-bc34-db9a439dee7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:31.354497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:31.355527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 319/631] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:31.356514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fea964-845a-49f6-bc34-db9a439dee7e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.365824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20490df-21cf-4075-a5de-9c95896b5778 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:31.367499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20490df-21cf-4075-a5de-9c95896b5778 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-c20490df-21cf-4075-a5de-9c95896b5778 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:31.382419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:31.383543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 313/632] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:31.384691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20490df-21cf-4075-a5de-9c95896b5778 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.426252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:31.474928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fea964-845a-49f6-bc34-db9a439dee7e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.492525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20490df-21cf-4075-a5de-9c95896b5778 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.509419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:31.514760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34536388-c7dd-416e-a1b0-57ce4406feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.515572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536388-c7dd-416e-a1b0-57ce4406feea None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.515994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536388-c7dd-416e-a1b0-57ce4406feea None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.516421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34536388-c7dd-416e-a1b0-57ce4406feea None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.528827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34536388-c7dd-416e-a1b0-57ce4406feea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:31.529361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd9297-0260-4aa1-a6d3-12a1318e949d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.529968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd9297-0260-4aa1-a6d3-12a1318e949d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.530245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd9297-0260-4aa1-a6d3-12a1318e949d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.530578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd9297-0260-4aa1-a6d3-12a1318e949d None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.531244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd9297-0260-4aa1-a6d3-12a1318e949d None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:31.531594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd9297-0260-4aa1-a6d3-12a1318e949d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:31.533083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd9297-0260-4aa1-a6d3-12a1318e949d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-34536388-c7dd-416e-a1b0-57ce4406feea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:31.535547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:31.536567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:31.537507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:31.539453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 320/633] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:31.540366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34536388-c7dd-416e-a1b0-57ce4406feea None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.551237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 314/634] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:31.551237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd9297-0260-4aa1-a6d3-12a1318e949d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.658394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd9297-0260-4aa1-a6d3-12a1318e949d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.679886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34536388-c7dd-416e-a1b0-57ce4406feea None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.700105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:31.701945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cc417-8579-4af0-ba94-6857040723c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.702855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cc417-8579-4af0-ba94-6857040723c3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.703276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cc417-8579-4af0-ba94-6857040723c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.703754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cc417-8579-4af0-ba94-6857040723c3 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.704997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cc417-8579-4af0-ba94-6857040723c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:31.705437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cc417-8579-4af0-ba94-6857040723c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:31.707946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cc417-8579-4af0-ba94-6857040723c3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:31.717631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 315/635] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:31.718698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cc417-8579-4af0-ba94-6857040723c3 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.735169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec58c2-7d14-4762-a723-26127caca2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.736193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec58c2-7d14-4762-a723-26127caca2ff None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.736578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec58c2-7d14-4762-a723-26127caca2ff None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.737018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec58c2-7d14-4762-a723-26127caca2ff None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.738419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec58c2-7d14-4762-a723-26127caca2ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:31.738795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec58c2-7d14-4762-a723-26127caca2ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:31.740897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec58c2-7d14-4762-a723-26127caca2ff None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:31.765761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 321/636] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:31.766782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec58c2-7d14-4762-a723-26127caca2ff None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.767985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:31.835714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cc417-8579-4af0-ba94-6857040723c3 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.873002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cd1b-ebd2-4a9e-a573-d1023f53599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.873632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cd1b-ebd2-4a9e-a573-d1023f53599e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.873911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cd1b-ebd2-4a9e-a573-d1023f53599e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.874285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223cd1b-ebd2-4a9e-a573-d1023f53599e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.886433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223cd1b-ebd2-4a9e-a573-d1023f53599e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:31.888100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e223cd1b-ebd2-4a9e-a573-d1023f53599e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:31.891058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec58c2-7d14-4762-a723-26127caca2ff None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-e223cd1b-ebd2-4a9e-a573-d1023f53599e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:31.903492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:31.904439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:31.905429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:31.905429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:31.905429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:31.905429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 316/637] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:31.906433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cd1b-ebd2-4a9e-a573-d1023f53599e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.928751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4485876-d92d-4ff0-a1da-28dda28196cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:31.929306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4485876-d92d-4ff0-a1da-28dda28196cb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:31.929582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4485876-d92d-4ff0-a1da-28dda28196cb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:31.929850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4485876-d92d-4ff0-a1da-28dda28196cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:31.941173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4485876-d92d-4ff0-a1da-28dda28196cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:31.942785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4485876-d92d-4ff0-a1da-28dda28196cb None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-f4485876-d92d-4ff0-a1da-28dda28196cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:31.960511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:31.961573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:31.962898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:31.962898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:31.962898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:31.966355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 322/638] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:31.967289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4485876-d92d-4ff0-a1da-28dda28196cb None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:31.997400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:32.011153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:32.022134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223cd1b-ebd2-4a9e-a573-d1023f53599e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:32.057981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9ec16-a5d2-444b-860f-c69e4de18784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:32.058584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9ec16-a5d2-444b-860f-c69e4de18784 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.058857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9ec16-a5d2-444b-860f-c69e4de18784 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.059130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9ec16-a5d2-444b-860f-c69e4de18784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.082708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4485876-d92d-4ff0-a1da-28dda28196cb None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:32.119597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae314b76-9c8a-4bcf-b023-e83fc2e1b8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:32.120620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae314b76-9c8a-4bcf-b023-e83fc2e1b8d5 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.121242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae314b76-9c8a-4bcf-b023-e83fc2e1b8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.121588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae314b76-9c8a-4bcf-b023-e83fc2e1b8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.122229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae314b76-9c8a-4bcf-b023-e83fc2e1b8d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:32.122531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae314b76-9c8a-4bcf-b023-e83fc2e1b8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:32.123890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae314b76-9c8a-4bcf-b023-e83fc2e1b8d5 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:32.128549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9ec16-a5d2-444b-860f-c69e4de18784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:32.130014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9ec16-a5d2-444b-860f-c69e4de18784 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:32.141107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 323/639] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:32.141107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae314b76-9c8a-4bcf-b023-e83fc2e1b8d5 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:32.231031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:32.238821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:32.255066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 317/640] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:32.255066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6576248-d1d3-468a-a6cf-3efcf3953251 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.255066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6576248-d1d3-468a-a6cf-3efcf3953251 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.255066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6576248-d1d3-468a-a6cf-3efcf3953251 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.255066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 318/641] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:32.255066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6576248-d1d3-468a-a6cf-3efcf3953251 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:32.255990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae314b76-9c8a-4bcf-b023-e83fc2e1b8d5 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:32.297187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63548dbc-2e26-4e34-b0d3-9118ddb5bc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:32.298581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63548dbc-2e26-4e34-b0d3-9118ddb5bc9a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.299077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63548dbc-2e26-4e34-b0d3-9118ddb5bc9a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.299539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63548dbc-2e26-4e34-b0d3-9118ddb5bc9a None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.300483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63548dbc-2e26-4e34-b0d3-9118ddb5bc9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:32.301002 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63548dbc-2e26-4e34-b0d3-9118ddb5bc9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:32.302580 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63548dbc-2e26-4e34-b0d3-9118ddb5bc9a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:32.314896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 324/642] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:32.316333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63548dbc-2e26-4e34-b0d3-9118ddb5bc9a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:32.370707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6576248-d1d3-468a-a6cf-3efcf3953251 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:32.406853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c0556-1885-4745-9439-3a6ca482dc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:32.407472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c0556-1885-4745-9439-3a6ca482dc4a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.407745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c0556-1885-4745-9439-3a6ca482dc4a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.408010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c0556-1885-4745-9439-3a6ca482dc4a None admin] PATH_INFO: `/v3/users/glance` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.415513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c0556-1885-4745-9439-3a6ca482dc4a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:32.416969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c0556-1885-4745-9439-3a6ca482dc4a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:32.428888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63548dbc-2e26-4e34-b0d3-9118ddb5bc9a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-1f2c0556-1885-4745-9439-3a6ca482dc4a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:32.429385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:32.430332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:32.431258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 319/643] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:32.431870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5102f48-c695-4cc0-8f06-53b3d6513bbf None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.432436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5102f48-c695-4cc0-8f06-53b3d6513bbf None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.432436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5102f48-c695-4cc0-8f06-53b3d6513bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.449935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f5102f48-c695-4cc0-8f06-53b3d6513bbf None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:32.451113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f5102f48-c695-4cc0-8f06-53b3d6513bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:32.469622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372aee5-6bee-4cb5-8f4b-095a66936f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:32.470243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372aee5-6bee-4cb5-8f4b-095a66936f1e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.470541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372aee5-6bee-4cb5-8f4b-095a66936f1e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.470783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372aee5-6bee-4cb5-8f4b-095a66936f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.482809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372aee5-6bee-4cb5-8f4b-095a66936f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:32.484181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372aee5-6bee-4cb5-8f4b-095a66936f1e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:32.486999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-3372aee5-6bee-4cb5-8f4b-095a66936f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:32.500449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:32.501441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:32.507898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:32.507898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:32.507898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:32.507898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 325/644] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:32.507898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372aee5-6bee-4cb5-8f4b-095a66936f1e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:32.509129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:32.535995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 320/645] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:32.535995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5102f48-c695-4cc0-8f06-53b3d6513bbf None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:32.635008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372aee5-6bee-4cb5-8f4b-095a66936f1e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:32.650594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5102f48-c695-4cc0-8f06-53b3d6513bbf None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:32.683574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec81950-43a0-4b07-99a9-b50a0916a6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:32.684636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec81950-43a0-4b07-99a9-b50a0916a6c2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.685008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec81950-43a0-4b07-99a9-b50a0916a6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.685416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec81950-43a0-4b07-99a9-b50a0916a6c2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.689129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f719ea0-b3a4-4e35-beb6-81820012b0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:32.689898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f719ea0-b3a4-4e35-beb6-81820012b0d8 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.690429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f719ea0-b3a4-4e35-beb6-81820012b0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.690849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f719ea0-b3a4-4e35-beb6-81820012b0d8 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.691886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f719ea0-b3a4-4e35-beb6-81820012b0d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:32.692366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f719ea0-b3a4-4e35-beb6-81820012b0d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:32.694278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f719ea0-b3a4-4e35-beb6-81820012b0d8 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:32.694398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec81950-43a0-4b07-99a9-b50a0916a6c2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:32.695911 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec81950-43a0-4b07-99a9-b50a0916a6c2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-aec81950-43a0-4b07-99a9-b50a0916a6c2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:32.711287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:32.712466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 00:17:32.713647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:32.715753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 326/646] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:32.716742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec81950-43a0-4b07-99a9-b50a0916a6c2 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:32.717760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 321/647] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 00:17:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:32.723899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f719ea0-b3a4-4e35-beb6-81820012b0d8 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:32.774447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:32.795759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:32.842117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f719ea0-b3a4-4e35-beb6-81820012b0d8 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:32.846465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec81950-43a0-4b07-99a9-b50a0916a6c2 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:32.882494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f20c7-1757-4c6f-8db4-ce621a51f651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:32.882772 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e5bf3-30f3-43c1-a8cd-c54e85c1b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:32.883102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f20c7-1757-4c6f-8db4-ce621a51f651 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.883380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f20c7-1757-4c6f-8db4-ce621a51f651 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.883477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e5bf3-30f3-43c1-a8cd-c54e85c1b702 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:32.883659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f20c7-1757-4c6f-8db4-ce621a51f651 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.883752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e5bf3-30f3-43c1-a8cd-c54e85c1b702 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:32.883974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e5bf3-30f3-43c1-a8cd-c54e85c1b702 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:32.884372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f20c7-1757-4c6f-8db4-ce621a51f651 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:32.884650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e5bf3-30f3-43c1-a8cd-c54e85c1b702 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:32.884757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f20c7-1757-4c6f-8db4-ce621a51f651 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:32.884956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e5bf3-30f3-43c1-a8cd-c54e85c1b702 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:32.886121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f20c7-1757-4c6f-8db4-ce621a51f651 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:32.886314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e5bf3-30f3-43c1-a8cd-c54e85c1b702 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:32.903362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 327/648] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:32.903362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e5bf3-30f3-43c1-a8cd-c54e85c1b702 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:32.907492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 322/649] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 1 00:17:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:32.908314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f20c7-1757-4c6f-8db4-ce621a51f651 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.009559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.013756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f20c7-1757-4c6f-8db4-ce621a51f651 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.014629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e5bf3-30f3-43c1-a8cd-c54e85c1b702 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.033920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.052384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bba968-dfb7-4875-8dac-c69b43e5ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.053047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bba968-dfb7-4875-8dac-c69b43e5ebff None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.053342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bba968-dfb7-4875-8dac-c69b43e5ebff None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.053589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bba968-dfb7-4875-8dac-c69b43e5ebff None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.053946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f67d1-2c1e-4b71-8fc2-1cf0bd8af731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.054548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f67d1-2c1e-4b71-8fc2-1cf0bd8af731 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.054828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f67d1-2c1e-4b71-8fc2-1cf0bd8af731 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.055105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f67d1-2c1e-4b71-8fc2-1cf0bd8af731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.064875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bba968-dfb7-4875-8dac-c69b43e5ebff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-15bba968-dfb7-4875-8dac-c69b43e5ebff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:33.069908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:33.070862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:33.071703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 323/650] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:33.072404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bba968-dfb7-4875-8dac-c69b43e5ebff None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.136242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f67d1-2c1e-4b71-8fc2-1cf0bd8af731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:33.137481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f67d1-2c1e-4b71-8fc2-1cf0bd8af731 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:33.186877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bba968-dfb7-4875-8dac-c69b43e5ebff None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.219882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24402607-9776-454b-973e-afcaac4d37f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.220511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24402607-9776-454b-973e-afcaac4d37f6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.220797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24402607-9776-454b-973e-afcaac4d37f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.221065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24402607-9776-454b-973e-afcaac4d37f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.228674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24402607-9776-454b-973e-afcaac4d37f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:33.230315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24402607-9776-454b-973e-afcaac4d37f6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:33.240508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 328/651] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:33.241362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f67d1-2c1e-4b71-8fc2-1cf0bd8af731 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-24402607-9776-454b-973e-afcaac4d37f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:33.242385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:33.243239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:33.244198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:33.244198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:33.244198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:33.244198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:33.244198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:33.244198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:33.244198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:33.244198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:33.244198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 324/652] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:33.245172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24402607-9776-454b-973e-afcaac4d37f6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.261708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.265163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.343688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f67d1-2c1e-4b71-8fc2-1cf0bd8af731 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.344214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24402607-9776-454b-973e-afcaac4d37f6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.377469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51baf8b5-2766-4951-8246-d6e11f3088f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.378671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ffd52-2a7d-4bba-87fc-0c595b531491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.378768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baf8b5-2766-4951-8246-d6e11f3088f3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.378855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ffd52-2a7d-4bba-87fc-0c595b531491 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.378927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baf8b5-2766-4951-8246-d6e11f3088f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.379003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ffd52-2a7d-4bba-87fc-0c595b531491 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.379183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baf8b5-2766-4951-8246-d6e11f3088f3 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.379275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ffd52-2a7d-4bba-87fc-0c595b531491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.379801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51baf8b5-2766-4951-8246-d6e11f3088f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:33.380099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51baf8b5-2766-4951-8246-d6e11f3088f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:33.381514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51baf8b5-2766-4951-8246-d6e11f3088f3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:33.389272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 325/653] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:33.390572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb175f7-e33f-4c93-b92c-7d596cf2d7eb None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.390857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb175f7-e33f-4c93-b92c-7d596cf2d7eb None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.391123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb175f7-e33f-4c93-b92c-7d596cf2d7eb None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.392425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 326/654] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:33.393078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb175f7-e33f-4c93-b92c-7d596cf2d7eb None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.450881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ffd52-2a7d-4bba-87fc-0c595b531491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:33.452338 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ffd52-2a7d-4bba-87fc-0c595b531491 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:33.469149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.472586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.494248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb175f7-e33f-4c93-b92c-7d596cf2d7eb None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.527985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8633591-fda9-477c-92fd-c6ecd7680970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.528636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8633591-fda9-477c-92fd-c6ecd7680970 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.528911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8633591-fda9-477c-92fd-c6ecd7680970 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.529187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8633591-fda9-477c-92fd-c6ecd7680970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.539232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8633591-fda9-477c-92fd-c6ecd7680970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:33.540706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8633591-fda9-477c-92fd-c6ecd7680970 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:33.554469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 329/655] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:33.554469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ffd52-2a7d-4bba-87fc-0c595b531491 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-b8633591-fda9-477c-92fd-c6ecd7680970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:33.555590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:33.556488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:33.557418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:33.557418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:33.557418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:33.557418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 327/656] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:33.557640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8633591-fda9-477c-92fd-c6ecd7680970 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.650876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ffd52-2a7d-4bba-87fc-0c595b531491 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.660692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8633591-fda9-477c-92fd-c6ecd7680970 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.668162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.686212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1e97c-4769-4b94-9e8d-e2a26b9622fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.686815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1e97c-4769-4b94-9e8d-e2a26b9622fe None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.687089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1e97c-4769-4b94-9e8d-e2a26b9622fe None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.687356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1e97c-4769-4b94-9e8d-e2a26b9622fe None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.688035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1e97c-4769-4b94-9e8d-e2a26b9622fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:33.688334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1e97c-4769-4b94-9e8d-e2a26b9622fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:33.689565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.689802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1e97c-4769-4b94-9e8d-e2a26b9622fe None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:33.695182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9e5af-03a0-4d14-afd5-3190bf3542db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.695808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9e5af-03a0-4d14-afd5-3190bf3542db None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.696084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9e5af-03a0-4d14-afd5-3190bf3542db None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.696389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9e5af-03a0-4d14-afd5-3190bf3542db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.701217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 330/657] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:33.702385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd82f64-8a14-4b3a-8fbd-ba74e07e9893 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.702662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd82f64-8a14-4b3a-8fbd-ba74e07e9893 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.702938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd82f64-8a14-4b3a-8fbd-ba74e07e9893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.708025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9e5af-03a0-4d14-afd5-3190bf3542db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:33.709512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9e5af-03a0-4d14-afd5-3190bf3542db None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:33.721459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ecd82f64-8a14-4b3a-8fbd-ba74e07e9893 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:33.723341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ecd82f64-8a14-4b3a-8fbd-ba74e07e9893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-04c9e5af-03a0-4d14-afd5-3190bf3542db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:33.724411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:33.725778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:33.727211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:33.727211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:33.727211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:33.727211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 328/658] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:33.727211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9e5af-03a0-4d14-afd5-3190bf3542db None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.793460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 331/659] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:33.794596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd82f64-8a14-4b3a-8fbd-ba74e07e9893 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.839617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9e5af-03a0-4d14-afd5-3190bf3542db None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.874621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962ab63-3d6d-4480-b9e3-bd6937718518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.875244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962ab63-3d6d-4480-b9e3-bd6937718518 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.875518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962ab63-3d6d-4480-b9e3-bd6937718518 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.875779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962ab63-3d6d-4480-b9e3-bd6937718518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.887601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962ab63-3d6d-4480-b9e3-bd6937718518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:33.889221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962ab63-3d6d-4480-b9e3-bd6937718518 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:33.897552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd82f64-8a14-4b3a-8fbd-ba74e07e9893 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-5962ab63-3d6d-4480-b9e3-bd6937718518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:33.904808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:33.905704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:33.906622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:33.906622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:33.906622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:33.906622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 329/660] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:33.906622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962ab63-3d6d-4480-b9e3-bd6937718518 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:33.908175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.940015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569fb95-c75d-4006-8a4c-2a19b4eba8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:33.941199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569fb95-c75d-4006-8a4c-2a19b4eba8bb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:33.941490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569fb95-c75d-4006-8a4c-2a19b4eba8bb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:33.941754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569fb95-c75d-4006-8a4c-2a19b4eba8bb None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:33.942391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569fb95-c75d-4006-8a4c-2a19b4eba8bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:33.942692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569fb95-c75d-4006-8a4c-2a19b4eba8bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:33.944069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569fb95-c75d-4006-8a4c-2a19b4eba8bb None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:33.945589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:33.961158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 332/661] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:33.961911 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569fb95-c75d-4006-8a4c-2a19b4eba8bb None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.016332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962ab63-3d6d-4480-b9e3-bd6937718518 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.051577 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1958da-2631-46ab-9d99-daab32bee53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.052271 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1958da-2631-46ab-9d99-daab32bee53e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.052570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1958da-2631-46ab-9d99-daab32bee53e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.052844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1958da-2631-46ab-9d99-daab32bee53e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.053561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1958da-2631-46ab-9d99-daab32bee53e None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=fb204040ab2f4168b35c3e78bacb849f) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:34.053862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1958da-2631-46ab-9d99-daab32bee53e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.055275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1958da-2631-46ab-9d99-daab32bee53e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.068563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 330/662] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/role_assignments?user.id=fb204040ab2f4168b35c3e78bacb849f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:34.068563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1958da-2631-46ab-9d99-daab32bee53e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.079103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569fb95-c75d-4006-8a4c-2a19b4eba8bb None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.124439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfe7cd-114c-4717-bb55-ee00bd8a69d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.125165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfe7cd-114c-4717-bb55-ee00bd8a69d9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.125470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfe7cd-114c-4717-bb55-ee00bd8a69d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.125684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfe7cd-114c-4717-bb55-ee00bd8a69d9 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.126372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfe7cd-114c-4717-bb55-ee00bd8a69d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:34.126671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfe7cd-114c-4717-bb55-ee00bd8a69d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.128539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfe7cd-114c-4717-bb55-ee00bd8a69d9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.143117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 333/663] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:34.143117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfe7cd-114c-4717-bb55-ee00bd8a69d9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.172323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:34.176655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1958da-2631-46ab-9d99-daab32bee53e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.180411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:34.216803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a26fd7-a901-41f9-a5e7-c6a0bfa6e93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.217426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a26fd7-a901-41f9-a5e7-c6a0bfa6e93c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.217785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a26fd7-a901-41f9-a5e7-c6a0bfa6e93c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.217960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a26fd7-a901-41f9-a5e7-c6a0bfa6e93c None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.219018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a26fd7-a901-41f9-a5e7-c6a0bfa6e93c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:34.219434 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a26fd7-a901-41f9-a5e7-c6a0bfa6e93c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.220851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a26fd7-a901-41f9-a5e7-c6a0bfa6e93c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.235208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 331/664] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:34.236043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a26fd7-a901-41f9-a5e7-c6a0bfa6e93c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.277573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfe7cd-114c-4717-bb55-ee00bd8a69d9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.314147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7851-ff74-49f2-a6b9-9d64033b3441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.314774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7851-ff74-49f2-a6b9-9d64033b3441 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.315051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7851-ff74-49f2-a6b9-9d64033b3441 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.315401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a7851-ff74-49f2-a6b9-9d64033b3441 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.322657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a7851-ff74-49f2-a6b9-9d64033b3441 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.324093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a7851-ff74-49f2-a6b9-9d64033b3441 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-877a7851-ff74-49f2-a6b9-9d64033b3441 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:34.336638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:34.337531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:34.338451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:34.338451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:34.338451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:34.338451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:34.338451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:34.338451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:34.338451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:34.338451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:34.338451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 334/665] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:34.339522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7851-ff74-49f2-a6b9-9d64033b3441 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.356600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a26fd7-a901-41f9-a5e7-c6a0bfa6e93c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.398038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8e166-c233-4d89-a5cd-0c432cb06357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.398944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8e166-c233-4d89-a5cd-0c432cb06357 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.399225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8e166-c233-4d89-a5cd-0c432cb06357 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.399661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8e166-c233-4d89-a5cd-0c432cb06357 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.410700 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8e166-c233-4d89-a5cd-0c432cb06357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-48c8e166-c233-4d89-a5cd-0c432cb06357 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:34.415554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:34.416485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:34.417372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:34.421419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 332/666] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:34.422182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8e166-c233-4d89-a5cd-0c432cb06357 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.432341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:34.447892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:34.450612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a7851-ff74-49f2-a6b9-9d64033b3441 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.492328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba0a76-601e-42ff-be21-105568015f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.493007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba0a76-601e-42ff-be21-105568015f97 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.493302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba0a76-601e-42ff-be21-105568015f97 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.493559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba0a76-601e-42ff-be21-105568015f97 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.501963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba0a76-601e-42ff-be21-105568015f97 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.504612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba0a76-601e-42ff-be21-105568015f97 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-6bba0a76-601e-42ff-be21-105568015f97 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:34.515977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:34.517109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 335/667] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:34.518034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba0a76-601e-42ff-be21-105568015f97 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.539113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8e166-c233-4d89-a5cd-0c432cb06357 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.576221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d471b-9213-4b75-81e4-5d9d2c99733e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.576909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d471b-9213-4b75-81e4-5d9d2c99733e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.577189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d471b-9213-4b75-81e4-5d9d2c99733e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.577461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d471b-9213-4b75-81e4-5d9d2c99733e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.585938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d471b-9213-4b75-81e4-5d9d2c99733e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.587643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d471b-9213-4b75-81e4-5d9d2c99733e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-4d2d471b-9213-4b75-81e4-5d9d2c99733e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:34.599229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:34.600393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 333/668] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:34.606753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d471b-9213-4b75-81e4-5d9d2c99733e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.636006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba0a76-601e-42ff-be21-105568015f97 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.673782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22aa69-a733-4c7b-b290-6ef42a3be85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.675068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22aa69-a733-4c7b-b290-6ef42a3be85a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.675348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22aa69-a733-4c7b-b290-6ef42a3be85a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.675708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22aa69-a733-4c7b-b290-6ef42a3be85a None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.676397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22aa69-a733-4c7b-b290-6ef42a3be85a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:34.676746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22aa69-a733-4c7b-b290-6ef42a3be85a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.678218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22aa69-a733-4c7b-b290-6ef42a3be85a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.696233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 336/669] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:34.696233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22aa69-a733-4c7b-b290-6ef42a3be85a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.699590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:34.707208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:34.720241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d471b-9213-4b75-81e4-5d9d2c99733e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.755862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503bcbc-3fa1-4d19-be48-31a49d6b12ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.756461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503bcbc-3fa1-4d19-be48-31a49d6b12ce None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.756770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503bcbc-3fa1-4d19-be48-31a49d6b12ce None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.757044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503bcbc-3fa1-4d19-be48-31a49d6b12ce None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.757709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503bcbc-3fa1-4d19-be48-31a49d6b12ce None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:34.758011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503bcbc-3fa1-4d19-be48-31a49d6b12ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.759423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503bcbc-3fa1-4d19-be48-31a49d6b12ce None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.767468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 334/670] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:34.772981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503bcbc-3fa1-4d19-be48-31a49d6b12ce None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.806896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22aa69-a733-4c7b-b290-6ef42a3be85a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.843071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed654b-1ec9-4577-962c-c67576ba109c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.844043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed654b-1ec9-4577-962c-c67576ba109c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.844043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed654b-1ec9-4577-962c-c67576ba109c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.844230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed654b-1ec9-4577-962c-c67576ba109c None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.844933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed654b-1ec9-4577-962c-c67576ba109c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:34.845236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed654b-1ec9-4577-962c-c67576ba109c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.847166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed654b-1ec9-4577-962c-c67576ba109c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.865649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 337/671] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:34.866360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed654b-1ec9-4577-962c-c67576ba109c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.881094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503bcbc-3fa1-4d19-be48-31a49d6b12ce None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:34.920254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3573ea-056d-4807-a9ab-d1a5e3961e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:34.920896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3573ea-056d-4807-a9ab-d1a5e3961e56 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:34.921169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3573ea-056d-4807-a9ab-d1a5e3961e56 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:34.921442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3573ea-056d-4807-a9ab-d1a5e3961e56 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:34.922139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3573ea-056d-4807-a9ab-d1a5e3961e56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:34.922498 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3573ea-056d-4807-a9ab-d1a5e3961e56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:34.923987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3573ea-056d-4807-a9ab-d1a5e3961e56 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:34.931127 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:34.944005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 335/672] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:34.944005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3573ea-056d-4807-a9ab-d1a5e3961e56 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:34.944005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:34.989681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed654b-1ec9-4577-962c-c67576ba109c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:35.026355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dc507-2adb-4c74-8df9-d6dc605d4a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:35.027097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dc507-2adb-4c74-8df9-d6dc605d4a56 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:35.027476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dc507-2adb-4c74-8df9-d6dc605d4a56 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:35.027869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dc507-2adb-4c74-8df9-d6dc605d4a56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:35.028728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dc507-2adb-4c74-8df9-d6dc605d4a56 None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=52470b620abd4cd29ab7e4752f285bb3) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:35.029215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dc507-2adb-4c74-8df9-d6dc605d4a56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:35.030720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dc507-2adb-4c74-8df9-d6dc605d4a56 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:35.044532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 338/673] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/role_assignments?user.id=52470b620abd4cd29ab7e4752f285bb3&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:35.044532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dc507-2adb-4c74-8df9-d6dc605d4a56 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:35.062943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3573ea-056d-4807-a9ab-d1a5e3961e56 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:35.103202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f56081-b4a4-46e1-80e5-dd7cf64c281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:35.103946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f56081-b4a4-46e1-80e5-dd7cf64c281b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:35.104292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f56081-b4a4-46e1-80e5-dd7cf64c281b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:35.104775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f56081-b4a4-46e1-80e5-dd7cf64c281b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:35.117107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f56081-b4a4-46e1-80e5-dd7cf64c281b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:35.118673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f56081-b4a4-46e1-80e5-dd7cf64c281b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-35f56081-b4a4-46e1-80e5-dd7cf64c281b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:35.136187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:35.137329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:35.138298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:35.138298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:35.138298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:35.142467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 336/674] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:35.143928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f56081-b4a4-46e1-80e5-dd7cf64c281b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:35.165253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dc507-2adb-4c74-8df9-d6dc605d4a56 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:35.206592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a567b2-d54a-4af0-b673-a9be20828daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:35.207627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a567b2-d54a-4af0-b673-a9be20828daf None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:35.208195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a567b2-d54a-4af0-b673-a9be20828daf None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:35.208719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a567b2-d54a-4af0-b673-a9be20828daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:35.213634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:35.220173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:35.297586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f56081-b4a4-46e1-80e5-dd7cf64c281b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:35.306394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a567b2-d54a-4af0-b673-a9be20828daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:35.308098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a567b2-d54a-4af0-b673-a9be20828daf None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:35.341238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93a537-69c6-4e90-8ab2-57e7fd51a099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:35.342639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93a537-69c6-4e90-8ab2-57e7fd51a099 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:35.342913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93a537-69c6-4e90-8ab2-57e7fd51a099 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:35.343178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93a537-69c6-4e90-8ab2-57e7fd51a099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:35.344094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93a537-69c6-4e90-8ab2-57e7fd51a099 None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=bb5590d8961d47aa846117a236730d93) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:35.344210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93a537-69c6-4e90-8ab2-57e7fd51a099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:35.347188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93a537-69c6-4e90-8ab2-57e7fd51a099 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:35.353127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 337/675] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:35 2022] GET /identity/v3/role_assignments?user.id=bb5590d8961d47aa846117a236730d93&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:35.354539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93a537-69c6-4e90-8ab2-57e7fd51a099 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:35.445332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 339/676] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:35.451263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a567b2-d54a-4af0-b673-a9be20828daf None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:35.472302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93a537-69c6-4e90-8ab2-57e7fd51a099 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:35.504929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:35.515935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae56292-8696-4263-97ad-21a6324a52eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:35.517073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae56292-8696-4263-97ad-21a6324a52eb None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:35.517550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae56292-8696-4263-97ad-21a6324a52eb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:35.518007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae56292-8696-4263-97ad-21a6324a52eb None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:35.518921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae56292-8696-4263-97ad-21a6324a52eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:35.519407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae56292-8696-4263-97ad-21a6324a52eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:35.523420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae56292-8696-4263-97ad-21a6324a52eb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:35.531528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:35.542259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 338/677] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:35.557335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae56292-8696-4263-97ad-21a6324a52eb None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:35.649454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a567b2-d54a-4af0-b673-a9be20828daf None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:35.682627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae56292-8696-4263-97ad-21a6324a52eb None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:35.706731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0748-a3e0-4884-a728-8fdd3023535f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:35.708859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0748-a3e0-4884-a728-8fdd3023535f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:35.709626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0748-a3e0-4884-a728-8fdd3023535f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:35.710521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0748-a3e0-4884-a728-8fdd3023535f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:35.723999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6f477-ea3d-4a3b-8c93-51d1f78291a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:35.724675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6f477-ea3d-4a3b-8c93-51d1f78291a6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:35.724952 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6f477-ea3d-4a3b-8c93-51d1f78291a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:35.725478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6f477-ea3d-4a3b-8c93-51d1f78291a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:35.726661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f0748-a3e0-4884-a728-8fdd3023535f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:35.728757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f0748-a3e0-4884-a728-8fdd3023535f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:35.734100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6f477-ea3d-4a3b-8c93-51d1f78291a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:35.735536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae6f477-ea3d-4a3b-8c93-51d1f78291a6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-0d6f0748-a3e0-4884-a728-8fdd3023535f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:35.750481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:35.751480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:35.752339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:35.752339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:35.752339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-cae6f477-ea3d-4a3b-8c93-51d1f78291a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:35.752537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:35.753575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:35.754493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:35.755933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 339/678] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:35.757661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 340/679] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:35.758042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6f477-ea3d-4a3b-8c93-51d1f78291a6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:35.765246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0748-a3e0-4884-a728-8fdd3023535f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:35.849142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:35.891465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:35.926810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6f477-ea3d-4a3b-8c93-51d1f78291a6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:35.936835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0748-a3e0-4884-a728-8fdd3023535f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:35.968948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27a140-a6d9-4cd0-9d2c-0b598f4a94bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:35.970768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27a140-a6d9-4cd0-9d2c-0b598f4a94bc None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:35.972019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27a140-a6d9-4cd0-9d2c-0b598f4a94bc None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:35.973501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27a140-a6d9-4cd0-9d2c-0b598f4a94bc None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:35.977269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27a140-a6d9-4cd0-9d2c-0b598f4a94bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:35.978400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27a140-a6d9-4cd0-9d2c-0b598f4a94bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:35.980061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27a140-a6d9-4cd0-9d2c-0b598f4a94bc None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:36.008818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cf3b1-9282-4486-8276-aa12a77743ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:36.009761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cf3b1-9282-4486-8276-aa12a77743ee None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:36.010792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cf3b1-9282-4486-8276-aa12a77743ee None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:36.011289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cf3b1-9282-4486-8276-aa12a77743ee None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:36.013214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cf3b1-9282-4486-8276-aa12a77743ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:36.013953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cf3b1-9282-4486-8276-aa12a77743ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:36.015959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 340/680] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 00:17:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:36.017598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cf3b1-9282-4486-8276-aa12a77743ee None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:36.022057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27a140-a6d9-4cd0-9d2c-0b598f4a94bc None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:36.045642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 341/681] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:36.051611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cf3b1-9282-4486-8276-aa12a77743ee None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:36.223699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:36.227013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cf3b1-9282-4486-8276-aa12a77743ee None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:36.233995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27a140-a6d9-4cd0-9d2c-0b598f4a94bc None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:36.290387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d604e42-8e3c-4be8-ba7c-74103d0634aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:36.291008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d604e42-8e3c-4be8-ba7c-74103d0634aa None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:36.291285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d604e42-8e3c-4be8-ba7c-74103d0634aa None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:36.291556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d604e42-8e3c-4be8-ba7c-74103d0634aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:36.308289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b029b-ead6-4da5-914e-d754ed8ab460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:36.308962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b029b-ead6-4da5-914e-d754ed8ab460 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:36.309244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b029b-ead6-4da5-914e-d754ed8ab460 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:36.309517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b029b-ead6-4da5-914e-d754ed8ab460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:36.323446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b029b-ead6-4da5-914e-d754ed8ab460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:36.324954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b029b-ead6-4da5-914e-d754ed8ab460 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:36.332566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-cb8b029b-ead6-4da5-914e-d754ed8ab460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:36.345251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:36.346174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:36.347562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:36.347562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:36.347562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:36.347562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:36.347562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:36.347562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:36.347562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:36.347562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:36.347562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 342/682] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:36.348570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ebfc9-f096-4c6d-845a-af649bede162 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:36.348679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ebfc9-f096-4c6d-845a-af649bede162 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:36.348923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ebfc9-f096-4c6d-845a-af649bede162 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:36.350406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 343/683] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:36.355039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ebfc9-f096-4c6d-845a-af649bede162 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:36.397364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d604e42-8e3c-4be8-ba7c-74103d0634aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:36.398939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d604e42-8e3c-4be8-ba7c-74103d0634aa None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:36.487571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ebfc9-f096-4c6d-845a-af649bede162 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:36.500788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:36.513920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 341/684] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 00:17:36.515246 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbd49e-facb-4ad8-977e-7bd056d5f0f3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:36.515529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbd49e-facb-4ad8-977e-7bd056d5f0f3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:36.515897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbd49e-facb-4ad8-977e-7bd056d5f0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:36.536610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-57bbd49e-facb-4ad8-977e-7bd056d5f0f3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:36.537815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-57bbd49e-facb-4ad8-977e-7bd056d5f0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:36.540458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2bd36-a7a7-4d87-bdee-76b6a3a47a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:36.542005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2bd36-a7a7-4d87-bdee-76b6a3a47a6f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:36.542290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2bd36-a7a7-4d87-bdee-76b6a3a47a6f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:36.542559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2bd36-a7a7-4d87-bdee-76b6a3a47a6f None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:36.543253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2bd36-a7a7-4d87-bdee-76b6a3a47a6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:36.543555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2bd36-a7a7-4d87-bdee-76b6a3a47a6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:36.545133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2bd36-a7a7-4d87-bdee-76b6a3a47a6f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:36.562618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 344/685] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:36.563565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2bd36-a7a7-4d87-bdee-76b6a3a47a6f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:36.630107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 342/686] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:36.631048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbd49e-facb-4ad8-977e-7bd056d5f0f3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:36.647407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:36.716656 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2bd36-a7a7-4d87-bdee-76b6a3a47a6f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:36.786289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc0c35-5b40-48ee-8e49-7eb9f02865de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:36.786966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc0c35-5b40-48ee-8e49-7eb9f02865de None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:36.787223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc0c35-5b40-48ee-8e49-7eb9f02865de None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:36.787488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc0c35-5b40-48ee-8e49-7eb9f02865de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:36.789775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbd49e-facb-4ad8-977e-7bd056d5f0f3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:36.803032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc0c35-5b40-48ee-8e49-7eb9f02865de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:36.804509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc0c35-5b40-48ee-8e49-7eb9f02865de None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:36.828889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-d9dc0c35-5b40-48ee-8e49-7eb9f02865de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:36.835031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:36.835972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:36.837250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:36.837250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:36.837250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:36.841603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dc194-f973-4d71-9bc5-ffa7a76c5128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:36.842477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dc194-f973-4d71-9bc5-ffa7a76c5128 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:36.842965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dc194-f973-4d71-9bc5-ffa7a76c5128 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:36.843438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dc194-f973-4d71-9bc5-ffa7a76c5128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:36.844367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dc194-f973-4d71-9bc5-ffa7a76c5128 None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=cdf97e9e0340441c87316ce03d46cfac) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:36.846216 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dc194-f973-4d71-9bc5-ffa7a76c5128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:36.848674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dc194-f973-4d71-9bc5-ffa7a76c5128 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:36.850780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 345/687] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:36.852246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc0c35-5b40-48ee-8e49-7eb9f02865de None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:36.865446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 343/688] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:36 2022] GET /identity/v3/role_assignments?user.id=cdf97e9e0340441c87316ce03d46cfac&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:36.990689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc0c35-5b40-48ee-8e49-7eb9f02865de None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:37.009996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:37.038039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afd95f-9cd8-4299-826e-6879d40723ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:37.038574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afd95f-9cd8-4299-826e-6879d40723ac None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.038850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afd95f-9cd8-4299-826e-6879d40723ac None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.039118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afd95f-9cd8-4299-826e-6879d40723ac None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.039767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afd95f-9cd8-4299-826e-6879d40723ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:37.040066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afd95f-9cd8-4299-826e-6879d40723ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:37.041580 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5afd95f-9cd8-4299-826e-6879d40723ac None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:37.059686 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 346/689] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:37.065706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dc194-f973-4d71-9bc5-ffa7a76c5128 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:37.101804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afd95f-9cd8-4299-826e-6879d40723ac None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:37.211085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dc194-f973-4d71-9bc5-ffa7a76c5128 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:37.242012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afd95f-9cd8-4299-826e-6879d40723ac None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:37.252323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db59e6-3658-462e-9966-00ddc11df5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:37.253179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db59e6-3658-462e-9966-00ddc11df5a1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.253461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db59e6-3658-462e-9966-00ddc11df5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.253728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db59e6-3658-462e-9966-00ddc11df5a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.262565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db59e6-3658-462e-9966-00ddc11df5a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:37.264027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db59e6-3658-462e-9966-00ddc11df5a1 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-43db59e6-3658-462e-9966-00ddc11df5a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:37.281554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:37.282515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:37.283388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:37.283388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:37.283388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:37.283388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:37.283388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:37.283388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:37.283388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:37.283388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:37.283388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 344/690] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:37.285061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31d243-e101-4d08-8251-e6a2a14f5e3f None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.285061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31d243-e101-4d08-8251-e6a2a14f5e3f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.285223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31d243-e101-4d08-8251-e6a2a14f5e3f None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.290802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 345/691] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:37.290802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31d243-e101-4d08-8251-e6a2a14f5e3f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:37.296040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d5223-2364-4253-822e-b8d41ca75bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:37.297007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5223-2364-4253-822e-b8d41ca75bc9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.297509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5223-2364-4253-822e-b8d41ca75bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.297975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5223-2364-4253-822e-b8d41ca75bc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.310610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d5223-2364-4253-822e-b8d41ca75bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-852d5223-2364-4253-822e-b8d41ca75bc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:37.317075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:37.318036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:37.318877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:37.321820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 347/692] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:37.323207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f8234-2cb2-472f-bf86-520dbb306c13 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.323698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f8234-2cb2-472f-bf86-520dbb306c13 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.324186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f8234-2cb2-472f-bf86-520dbb306c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.343778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-c49f8234-2cb2-472f-bf86-520dbb306c13 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:37.345511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-c49f8234-2cb2-472f-bf86-520dbb306c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:37.353351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:37.379907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:37.416298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31d243-e101-4d08-8251-e6a2a14f5e3f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:37.440522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 348/693] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:37.440522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f8234-2cb2-472f-bf86-520dbb306c13 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:37.474090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00788058-2c3e-4e41-bf25-4a15973796da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:37.475346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00788058-2c3e-4e41-bf25-4a15973796da None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.475830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00788058-2c3e-4e41-bf25-4a15973796da None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.476310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00788058-2c3e-4e41-bf25-4a15973796da None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.477240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00788058-2c3e-4e41-bf25-4a15973796da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:37.477739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00788058-2c3e-4e41-bf25-4a15973796da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:37.479385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00788058-2c3e-4e41-bf25-4a15973796da None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:37.500319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 346/694] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:37.506047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00788058-2c3e-4e41-bf25-4a15973796da None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:37.549585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f8234-2cb2-472f-bf86-520dbb306c13 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:37.594249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cccae4-bf2a-4016-bd10-88b365e936ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:37.594859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cccae4-bf2a-4016-bd10-88b365e936ab None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.595134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cccae4-bf2a-4016-bd10-88b365e936ab None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.595404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cccae4-bf2a-4016-bd10-88b365e936ab None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.596062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cccae4-bf2a-4016-bd10-88b365e936ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:37.600517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cccae4-bf2a-4016-bd10-88b365e936ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:37.600517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cccae4-bf2a-4016-bd10-88b365e936ab None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:37.614402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 349/695] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:37.615695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cccae4-bf2a-4016-bd10-88b365e936ab None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:37.644797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:37.729264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cccae4-bf2a-4016-bd10-88b365e936ab None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:37.731403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00788058-2c3e-4e41-bf25-4a15973796da None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:37.754416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:37.769196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dee839-e7ae-42b2-96d7-de0447e7e79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:37.770636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dee839-e7ae-42b2-96d7-de0447e7e79e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.771071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dee839-e7ae-42b2-96d7-de0447e7e79e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.771492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dee839-e7ae-42b2-96d7-de0447e7e79e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.781538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fb14f-ef89-45ca-be8b-09525ee90f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:37.782145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fb14f-ef89-45ca-be8b-09525ee90f9c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.782419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fb14f-ef89-45ca-be8b-09525ee90f9c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.782688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fb14f-ef89-45ca-be8b-09525ee90f9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.783406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fb14f-ef89-45ca-be8b-09525ee90f9c None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=43d6c2a533e849cab240257cc1450c9f) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:37.784209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fb14f-ef89-45ca-be8b-09525ee90f9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:37.785668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fb14f-ef89-45ca-be8b-09525ee90f9c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:37.788836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dee839-e7ae-42b2-96d7-de0447e7e79e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:37.790522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dee839-e7ae-42b2-96d7-de0447e7e79e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:37.792867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 347/696] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:37 2022] GET /identity/v3/role_assignments?user.id=43d6c2a533e849cab240257cc1450c9f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:37.794567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48578456-f998-4cfb-8646-eec0150ef9fc None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.794852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48578456-f998-4cfb-8646-eec0150ef9fc None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.795126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48578456-f998-4cfb-8646-eec0150ef9fc None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.800614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 348/697] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:37.802297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335669bc-1518-4646-a2f3-650e5bd04cd4 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:37.802599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335669bc-1518-4646-a2f3-650e5bd04cd4 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:37.802871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335669bc-1518-4646-a2f3-650e5bd04cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-37dee839-e7ae-42b2-96d7-de0447e7e79e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:37.816886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:37.817871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:37.818778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:37.818778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:37.818778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:37.821120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 350/698] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:37.827310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dee839-e7ae-42b2-96d7-de0447e7e79e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:37.827560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-335669bc-1518-4646-a2f3-650e5bd04cd4 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:37.828821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-335669bc-1518-4646-a2f3-650e5bd04cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:37.924674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 349/699] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:37.926074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335669bc-1518-4646-a2f3-650e5bd04cd4 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:37.926693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:37.958517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dee839-e7ae-42b2-96d7-de0447e7e79e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:38.002410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42c5bc-5636-45f9-bbe2-825d724b5ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:38.003638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42c5bc-5636-45f9-bbe2-825d724b5ed5 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:38.003967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42c5bc-5636-45f9-bbe2-825d724b5ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:38.004751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42c5bc-5636-45f9-bbe2-825d724b5ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:38.005543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42c5bc-5636-45f9-bbe2-825d724b5ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:38.005964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42c5bc-5636-45f9-bbe2-825d724b5ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:38.007604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42c5bc-5636-45f9-bbe2-825d724b5ed5 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:38.028543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 351/700] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:38.034631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42c5bc-5636-45f9-bbe2-825d724b5ed5 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:38.079180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335669bc-1518-4646-a2f3-650e5bd04cd4 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:38.083797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:38.120891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf612ee-ba1f-45a8-98ba-a9cd85cb9fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:38.121705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf612ee-ba1f-45a8-98ba-a9cd85cb9fce None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:38.122047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf612ee-ba1f-45a8-98ba-a9cd85cb9fce None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:38.122380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf612ee-ba1f-45a8-98ba-a9cd85cb9fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:38.137330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf612ee-ba1f-45a8-98ba-a9cd85cb9fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:38.139269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf612ee-ba1f-45a8-98ba-a9cd85cb9fce None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-fbf612ee-ba1f-45a8-98ba-a9cd85cb9fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:38.156922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:38.157914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:38.158886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:38.158886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:38.158886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:38.158886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 350/701] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:38.166683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf612ee-ba1f-45a8-98ba-a9cd85cb9fce None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:38.174827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42c5bc-5636-45f9-bbe2-825d724b5ed5 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:38.218010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d4517-04f7-421c-9419-2edb092a4f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:38.218649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4517-04f7-421c-9419-2edb092a4f4b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:38.219391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4517-04f7-421c-9419-2edb092a4f4b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:38.219666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d4517-04f7-421c-9419-2edb092a4f4b None admin] PATH_INFO: `/v3/users/nova` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:38.223546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:38.230610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d4517-04f7-421c-9419-2edb092a4f4b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:38.232987 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d4517-04f7-421c-9419-2edb092a4f4b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-db8d4517-04f7-421c-9419-2edb092a4f4b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:38.250878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:38.252067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:38.253597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:38.254161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 352/702] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 00:17:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:38.259684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d4517-04f7-421c-9419-2edb092a4f4b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:38.302893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf612ee-ba1f-45a8-98ba-a9cd85cb9fce None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:38.344324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea12967-7277-4aec-8cc7-88eb03a51f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:38.345284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea12967-7277-4aec-8cc7-88eb03a51f03 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:38.345777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea12967-7277-4aec-8cc7-88eb03a51f03 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:38.346778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea12967-7277-4aec-8cc7-88eb03a51f03 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:38.347621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea12967-7277-4aec-8cc7-88eb03a51f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:38.348149 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea12967-7277-4aec-8cc7-88eb03a51f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:38.350488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea12967-7277-4aec-8cc7-88eb03a51f03 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:38.366468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 351/703] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:38.372056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea12967-7277-4aec-8cc7-88eb03a51f03 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:38.387389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d4517-04f7-421c-9419-2edb092a4f4b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:38.391879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:38.438937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2fc8e-4a21-4adf-969e-8530397d992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:38.439560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2fc8e-4a21-4adf-969e-8530397d992c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:38.439841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2fc8e-4a21-4adf-969e-8530397d992c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:38.440832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2fc8e-4a21-4adf-969e-8530397d992c None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:38.442102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2fc8e-4a21-4adf-969e-8530397d992c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:38.442571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2fc8e-4a21-4adf-969e-8530397d992c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:38.444388 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2fc8e-4a21-4adf-969e-8530397d992c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:38.466564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 353/704] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 00:17:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:38.467742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2fc8e-4a21-4adf-969e-8530397d992c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:38.497172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea12967-7277-4aec-8cc7-88eb03a51f03 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:38.534556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66d3d7-2907-47da-841d-9532ce5c2228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:38.536022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66d3d7-2907-47da-841d-9532ce5c2228 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:38.536516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66d3d7-2907-47da-841d-9532ce5c2228 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:38.537000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66d3d7-2907-47da-841d-9532ce5c2228 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:38.537795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66d3d7-2907-47da-841d-9532ce5c2228 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:38.539370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66d3d7-2907-47da-841d-9532ce5c2228 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:38.549445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-fe66d3d7-2907-47da-841d-9532ce5c2228 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:38.552232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:38.553716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 00:17:38.554897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:38.557476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 352/705] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 1 00:17:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:38.558556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66d3d7-2907-47da-841d-9532ce5c2228 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:38.621234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2fc8e-4a21-4adf-969e-8530397d992c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:38.655543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:38.671669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d27325-5ce5-40fd-bcb2-5e52c1c6a828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:38.672288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d27325-5ce5-40fd-bcb2-5e52c1c6a828 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:38.672637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d27325-5ce5-40fd-bcb2-5e52c1c6a828 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:38.672921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d27325-5ce5-40fd-bcb2-5e52c1c6a828 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:38.674007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d27325-5ce5-40fd-bcb2-5e52c1c6a828 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:38.675939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d27325-5ce5-40fd-bcb2-5e52c1c6a828 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:38.680597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66d3d7-2907-47da-841d-9532ce5c2228 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:38.712524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 354/706] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:17:38 2022] POST /identity/v3/users => generated 254 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:38.712524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d27325-5ce5-40fd-bcb2-5e52c1c6a828 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:38.754289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c07546-2dd9-47b5-9c35-938b8b0e554a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:38.755377 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c07546-2dd9-47b5-9c35-938b8b0e554a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:38.755853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c07546-2dd9-47b5-9c35-938b8b0e554a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:38.756347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c07546-2dd9-47b5-9c35-938b8b0e554a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:38.843945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d27325-5ce5-40fd-bcb2-5e52c1c6a828 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:38.860399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:38.891975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6b2a7-b3ea-49e3-8215-c9c8b096e204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:38.892607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6b2a7-b3ea-49e3-8215-c9c8b096e204 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:38.894337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6b2a7-b3ea-49e3-8215-c9c8b096e204 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:38.894729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6b2a7-b3ea-49e3-8215-c9c8b096e204 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:38.895458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6b2a7-b3ea-49e3-8215-c9c8b096e204 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:38.895776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6b2a7-b3ea-49e3-8215-c9c8b096e204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:38.897466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6b2a7-b3ea-49e3-8215-c9c8b096e204 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:38.903139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 355/707] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 00:17:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:38.910513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6b2a7-b3ea-49e3-8215-c9c8b096e204 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:38.929744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c07546-2dd9-47b5-9c35-938b8b0e554a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:38.933393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c07546-2dd9-47b5-9c35-938b8b0e554a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:39.033212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:39.054431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6b2a7-b3ea-49e3-8215-c9c8b096e204 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:39.060144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 353/708] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:39.068837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c07546-2dd9-47b5-9c35-938b8b0e554a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:39.110516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b45e1-5365-4283-be4f-3f070411ac96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:39.111137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b45e1-5365-4283-be4f-3f070411ac96 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:39.111419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b45e1-5365-4283-be4f-3f070411ac96 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:39.111696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b45e1-5365-4283-be4f-3f070411ac96 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:39.112468 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b45e1-5365-4283-be4f-3f070411ac96 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:39.112808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b45e1-5365-4283-be4f-3f070411ac96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:39.114202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b45e1-5365-4283-be4f-3f070411ac96 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:39.123246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 356/709] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 00:17:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:39.148345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed842f-6ee5-4ea4-8ef8-52c27b93ac81 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:39.148658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed842f-6ee5-4ea4-8ef8-52c27b93ac81 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:39.148866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed842f-6ee5-4ea4-8ef8-52c27b93ac81 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:39.150368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 357/710] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:39.156388 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1d47d-ee2e-463c-a5ff-41f650438349 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:39.156738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1d47d-ee2e-463c-a5ff-41f650438349 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:39.156969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1d47d-ee2e-463c-a5ff-41f650438349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:39.177727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-84c1d47d-ee2e-463c-a5ff-41f650438349 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:39.178997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-84c1d47d-ee2e-463c-a5ff-41f650438349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:39.223977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c07546-2dd9-47b5-9c35-938b8b0e554a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:39.224935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:39.279514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdcccb-6348-4b06-8275-500557491068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:39.279514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdcccb-6348-4b06-8275-500557491068 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:39.279514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdcccb-6348-4b06-8275-500557491068 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:39.280042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdcccb-6348-4b06-8275-500557491068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:39.285562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 358/711] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 00:17:39.292183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdcccb-6348-4b06-8275-500557491068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:39.293686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdcccb-6348-4b06-8275-500557491068 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f3cdcccb-6348-4b06-8275-500557491068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:39.310491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:39.311506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:39.312422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:39.312422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:39.312422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:39.312422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 354/712] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:39.317688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1d47d-ee2e-463c-a5ff-41f650438349 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:39.427024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1d47d-ee2e-463c-a5ff-41f650438349 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:39.463273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d95a-73a9-4b5d-b44a-28c6663d3f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:39.463885 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d95a-73a9-4b5d-b44a-28c6663d3f44 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:39.464164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d95a-73a9-4b5d-b44a-28c6663d3f44 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:39.464440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d95a-73a9-4b5d-b44a-28c6663d3f44 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:39.465119 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d95a-73a9-4b5d-b44a-28c6663d3f44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:39.465422 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d95a-73a9-4b5d-b44a-28c6663d3f44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:39.466804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d95a-73a9-4b5d-b44a-28c6663d3f44 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:39.481067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 359/713] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:39.486561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdcccb-6348-4b06-8275-500557491068 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:39.532852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:39.574336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d95a-73a9-4b5d-b44a-28c6663d3f44 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:39.615529 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:39.647604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdcccb-6348-4b06-8275-500557491068 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:39.707845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d95a-73a9-4b5d-b44a-28c6663d3f44 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:39.729230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d01b40-6aa0-43dc-ba17-d834337a1664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:39.729882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d01b40-6aa0-43dc-ba17-d834337a1664 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:39.730162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d01b40-6aa0-43dc-ba17-d834337a1664 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:39.730435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d01b40-6aa0-43dc-ba17-d834337a1664 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:39.738282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d01b40-6aa0-43dc-ba17-d834337a1664 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:39.739758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d01b40-6aa0-43dc-ba17-d834337a1664 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:39.751081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b731d-312f-467c-9201-2143c4cfde8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:39.755385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b731d-312f-467c-9201-2143c4cfde8b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:39.755904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b731d-312f-467c-9201-2143c4cfde8b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:39.756383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b731d-312f-467c-9201-2143c4cfde8b None admin] PATH_INFO: `/v3/services/network` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:39.757192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b731d-312f-467c-9201-2143c4cfde8b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:39.759058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b731d-312f-467c-9201-2143c4cfde8b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-16d01b40-6aa0-43dc-ba17-d834337a1664 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:39.772106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:39.773115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:39.774125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:39.774125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:39.774125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:39.774125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:39.774125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:39.774125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:39.774125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:39.774125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:39.779526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 355/714] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:39.785459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d01b40-6aa0-43dc-ba17-d834337a1664 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:39.931355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-577b731d-312f-467c-9201-2143c4cfde8b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:39.945307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:39.946404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:39.947281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 360/715] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:39 2022] GET /identity/v3/services/network => generated 88 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:39.953211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b731d-312f-467c-9201-2143c4cfde8b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:39.954997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d01b40-6aa0-43dc-ba17-d834337a1664 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:40.002259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d639b32-ed70-41d0-a641-7a42e1ab4980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:40.003542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d639b32-ed70-41d0-a641-7a42e1ab4980 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:40.003819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d639b32-ed70-41d0-a641-7a42e1ab4980 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:40.004086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d639b32-ed70-41d0-a641-7a42e1ab4980 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:40.004820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d639b32-ed70-41d0-a641-7a42e1ab4980 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:40.005125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d639b32-ed70-41d0-a641-7a42e1ab4980 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:40.006566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d639b32-ed70-41d0-a641-7a42e1ab4980 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:40.028017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 356/716] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:40.037663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d639b32-ed70-41d0-a641-7a42e1ab4980 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:40.131627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:40.141251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b731d-312f-467c-9201-2143c4cfde8b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:40.176181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d9824-1af7-438b-aee4-9a67bca45205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:40.176825 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d9824-1af7-438b-aee4-9a67bca45205 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:40.177095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d9824-1af7-438b-aee4-9a67bca45205 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:40.177363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d9824-1af7-438b-aee4-9a67bca45205 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:40.177981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d9824-1af7-438b-aee4-9a67bca45205 None admin] RBAC: Adding query filter params (name=network) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:40.178275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d9824-1af7-438b-aee4-9a67bca45205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:40.179655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d9824-1af7-438b-aee4-9a67bca45205 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:40.184430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 361/717] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:17:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:40.185703 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3fd08-c8b5-4637-92b0-2b9df2d184a4 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:40.185978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3fd08-c8b5-4637-92b0-2b9df2d184a4 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:40.186245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3fd08-c8b5-4637-92b0-2b9df2d184a4 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:40.187676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 362/718] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:40.189756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3fd08-c8b5-4637-92b0-2b9df2d184a4 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:40.197585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d639b32-ed70-41d0-a641-7a42e1ab4980 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:40.247592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94d3a9-e477-408d-800e-8d21f8c582a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:40.251579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94d3a9-e477-408d-800e-8d21f8c582a3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:40.251869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94d3a9-e477-408d-800e-8d21f8c582a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:40.252145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94d3a9-e477-408d-800e-8d21f8c582a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:40.253050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94d3a9-e477-408d-800e-8d21f8c582a3 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=ed93469862874c75951d95e28929c904, user.id=fb204040ab2f4168b35c3e78bacb849f) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:40.253301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94d3a9-e477-408d-800e-8d21f8c582a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:40.255169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94d3a9-e477-408d-800e-8d21f8c582a3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:40.263726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 357/719] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:40 2022] GET /identity/v3/role_assignments?user.id=fb204040ab2f4168b35c3e78bacb849f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:40.264951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82e19-e413-4b8b-b823-60b4d120955f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:40.265496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82e19-e413-4b8b-b823-60b4d120955f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:40.265774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82e19-e413-4b8b-b823-60b4d120955f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:40.291828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e7c82e19-e413-4b8b-b823-60b4d120955f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:40.293219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e7c82e19-e413-4b8b-b823-60b4d120955f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:40.308139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:40.315971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3fd08-c8b5-4637-92b0-2b9df2d184a4 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:40.368236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95080bdb-0b7b-4596-9bcc-5f57eed7f836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:40.369073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95080bdb-0b7b-4596-9bcc-5f57eed7f836 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:40.369350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95080bdb-0b7b-4596-9bcc-5f57eed7f836 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:40.369619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95080bdb-0b7b-4596-9bcc-5f57eed7f836 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:40.370252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95080bdb-0b7b-4596-9bcc-5f57eed7f836 None admin] RBAC: Adding query filter params (type=network) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:40.370629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95080bdb-0b7b-4596-9bcc-5f57eed7f836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:40.372652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95080bdb-0b7b-4596-9bcc-5f57eed7f836 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:40.378902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 363/720] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:17:40 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:40.399481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 358/721] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:40.718204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95080bdb-0b7b-4596-9bcc-5f57eed7f836 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:40.746835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:40.857209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95080bdb-0b7b-4596-9bcc-5f57eed7f836 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:40.921259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e5607-c9a2-4259-8e21-cb45c0d5a299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:40.921889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e5607-c9a2-4259-8e21-cb45c0d5a299 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:40.922165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e5607-c9a2-4259-8e21-cb45c0d5a299 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:40.922431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e5607-c9a2-4259-8e21-cb45c0d5a299 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:40.935778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e5607-c9a2-4259-8e21-cb45c0d5a299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-696e5607-c9a2-4259-8e21-cb45c0d5a299 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:40.940806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:40.941720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:40.942561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 364/722] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:40.947944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c82e19-e413-4b8b-b823-60b4d120955f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:41.096923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c82e19-e413-4b8b-b823-60b4d120955f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:41.147091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d41b7-a2e6-40c5-a973-5dc76b29e37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:41.147712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d41b7-a2e6-40c5-a973-5dc76b29e37b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:41.147986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d41b7-a2e6-40c5-a973-5dc76b29e37b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:41.148255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d41b7-a2e6-40c5-a973-5dc76b29e37b None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:41.148953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d41b7-a2e6-40c5-a973-5dc76b29e37b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:41.149251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d41b7-a2e6-40c5-a973-5dc76b29e37b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:41.150645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d41b7-a2e6-40c5-a973-5dc76b29e37b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:41.164623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 359/723] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:41.171373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e5607-c9a2-4259-8e21-cb45c0d5a299 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:41.305536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11ab58-ed6c-44fa-8ebe-8473373913bc None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:41.305849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11ab58-ed6c-44fa-8ebe-8473373913bc None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:41.306097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11ab58-ed6c-44fa-8ebe-8473373913bc None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:41.307691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 360/724] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:41.312809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f46d0-c274-49c6-8530-04df55cb8f65 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:41.313094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f46d0-c274-49c6-8530-04df55cb8f65 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:41.313376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f46d0-c274-49c6-8530-04df55cb8f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:41.319556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:41.334495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6c4f46d0-c274-49c6-8530-04df55cb8f65 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:41.335831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6c4f46d0-c274-49c6-8530-04df55cb8f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:41.338155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:41.362553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e5607-c9a2-4259-8e21-cb45c0d5a299 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:41.415698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 361/725] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:41.436854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826d95e-758d-4751-b8c3-5fe9f8877b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:41.437491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826d95e-758d-4751-b8c3-5fe9f8877b6f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:41.437769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826d95e-758d-4751-b8c3-5fe9f8877b6f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:41.438103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826d95e-758d-4751-b8c3-5fe9f8877b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:41.449048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826d95e-758d-4751-b8c3-5fe9f8877b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:41.450546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826d95e-758d-4751-b8c3-5fe9f8877b6f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-8826d95e-758d-4751-b8c3-5fe9f8877b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:41.473255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:41.474205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:41.475344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:41.475344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:41.475344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:41.475344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 365/726] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:41.480429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f46d0-c274-49c6-8530-04df55cb8f65 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:41.590905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f46d0-c274-49c6-8530-04df55cb8f65 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:41.605891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c45ac-3dfa-4839-b687-0a10b6123c68 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:41.606174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c45ac-3dfa-4839-b687-0a10b6123c68 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:41.606451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c45ac-3dfa-4839-b687-0a10b6123c68 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:41.607783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 366/727] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:41.613113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f736336-4dc1-4e3f-9efe-b9a5ed8b3c0e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:41.613336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f736336-4dc1-4e3f-9efe-b9a5ed8b3c0e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:41.613589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f736336-4dc1-4e3f-9efe-b9a5ed8b3c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:41.631569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85a141-239d-4a01-a412-433da89df525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:41.632241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85a141-239d-4a01-a412-433da89df525 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:41.632548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85a141-239d-4a01-a412-433da89df525 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:41.632826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85a141-239d-4a01-a412-433da89df525 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:41.633512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85a141-239d-4a01-a412-433da89df525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:41.633814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85a141-239d-4a01-a412-433da89df525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:41.635366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85a141-239d-4a01-a412-433da89df525 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:41.642285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:41.650370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 362/728] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:41.653550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3f736336-4dc1-4e3f-9efe-b9a5ed8b3c0e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:41.654899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3f736336-4dc1-4e3f-9efe-b9a5ed8b3c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:41.657735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85a141-239d-4a01-a412-433da89df525 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:41.801915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 367/729] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:41.804023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f736336-4dc1-4e3f-9efe-b9a5ed8b3c0e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:41.863617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85a141-239d-4a01-a412-433da89df525 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:41.885594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:41.939599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f736336-4dc1-4e3f-9efe-b9a5ed8b3c0e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:41.946086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6722e3-2a84-4460-ad78-3c2ed245f16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:41.946819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6722e3-2a84-4460-ad78-3c2ed245f16a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:41.947165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6722e3-2a84-4460-ad78-3c2ed245f16a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:41.947506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6722e3-2a84-4460-ad78-3c2ed245f16a None admin] PATH_INFO: `/v3/users/swift` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:41.969482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6722e3-2a84-4460-ad78-3c2ed245f16a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:41.971377 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6722e3-2a84-4460-ad78-3c2ed245f16a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:41.980984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284d345-33fd-46c6-a441-0015b3fd4e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:41.982467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284d345-33fd-46c6-a441-0015b3fd4e19 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:41.982933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284d345-33fd-46c6-a441-0015b3fd4e19 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:41.983384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284d345-33fd-46c6-a441-0015b3fd4e19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:41.995239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284d345-33fd-46c6-a441-0015b3fd4e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-0f6722e3-2a84-4460-ad78-3c2ed245f16a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:41.997650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:41.998596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:41.999535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 363/730] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-d284d345-33fd-46c6-a441-0015b3fd4e19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:42.003995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:42.004960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:42.005788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:42.006420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6722e3-2a84-4460-ad78-3c2ed245f16a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:42.008604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 368/731] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:42.014230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284d345-33fd-46c6-a441-0015b3fd4e19 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:42.082979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:42.134088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6722e3-2a84-4460-ad78-3c2ed245f16a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:42.172885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b5b18-a490-4fbe-b76a-b8847c5c2958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:42.173796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b5b18-a490-4fbe-b76a-b8847c5c2958 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.175171 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b5b18-a490-4fbe-b76a-b8847c5c2958 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.175636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b5b18-a490-4fbe-b76a-b8847c5c2958 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:42.177104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b5b18-a490-4fbe-b76a-b8847c5c2958 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:42.177852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b5b18-a490-4fbe-b76a-b8847c5c2958 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:42.179649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b5b18-a490-4fbe-b76a-b8847c5c2958 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:42.204696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 364/732] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 00:17:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:42.206629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b5b18-a490-4fbe-b76a-b8847c5c2958 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:42.215946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284d345-33fd-46c6-a441-0015b3fd4e19 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:42.254384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:42.269489 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41a286-894b-455d-bb44-98815bb67684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:42.270105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41a286-894b-455d-bb44-98815bb67684 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.270386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41a286-894b-455d-bb44-98815bb67684 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.270658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41a286-894b-455d-bb44-98815bb67684 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:42.271333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41a286-894b-455d-bb44-98815bb67684 None admin] RBAC: Adding query filter params (name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:42.271636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41a286-894b-455d-bb44-98815bb67684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:42.273608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41a286-894b-455d-bb44-98815bb67684 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:42.284777 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 369/733] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:42.285903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41a286-894b-455d-bb44-98815bb67684 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:42.335489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b5b18-a490-4fbe-b76a-b8847c5c2958 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:42.358541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:42.382646 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdb4b2-fbed-4f14-b284-6b76a89133eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:42.383345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdb4b2-fbed-4f14-b284-6b76a89133eb None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.383628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdb4b2-fbed-4f14-b284-6b76a89133eb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.383903 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdb4b2-fbed-4f14-b284-6b76a89133eb None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:42.384638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdb4b2-fbed-4f14-b284-6b76a89133eb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:42.386156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdb4b2-fbed-4f14-b284-6b76a89133eb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:42.407356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41a286-894b-455d-bb44-98815bb67684 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:42.412136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 365/734] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:17:42 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:42.412136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdb4b2-fbed-4f14-b284-6b76a89133eb None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:42.510342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94acac9-edf6-4ca7-ae60-cf401f56b8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:42.511470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94acac9-edf6-4ca7-ae60-cf401f56b8ca None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.512001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94acac9-edf6-4ca7-ae60-cf401f56b8ca None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.512470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94acac9-edf6-4ca7-ae60-cf401f56b8ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:42.615200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:42.663133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94acac9-edf6-4ca7-ae60-cf401f56b8ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:42.665085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94acac9-edf6-4ca7-ae60-cf401f56b8ca None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:42.666331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdb4b2-fbed-4f14-b284-6b76a89133eb None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:42.709784 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b4840-8f7d-4b6c-82b9-cf1fcf8025b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:42.710617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b4840-8f7d-4b6c-82b9-cf1fcf8025b9 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.710904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b4840-8f7d-4b6c-82b9-cf1fcf8025b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.711180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b4840-8f7d-4b6c-82b9-cf1fcf8025b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:42.725297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b4840-8f7d-4b6c-82b9-cf1fcf8025b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:42.726510 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b4840-8f7d-4b6c-82b9-cf1fcf8025b9 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-4e4b4840-8f7d-4b6c-82b9-cf1fcf8025b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:42.745422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:42.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:42.747348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:42.747348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:42.747348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:42.747348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 366/735] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:42.753418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b4840-8f7d-4b6c-82b9-cf1fcf8025b9 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:42.785048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:42.831900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 370/736] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:42 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 00:17:42.833285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eeb8d-f5b9-41e8-9bae-94a11714e543 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.833573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eeb8d-f5b9-41e8-9bae-94a11714e543 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.833846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eeb8d-f5b9-41e8-9bae-94a11714e543 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:42.835216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 371/737] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:42.836142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66b377-c1e4-431e-81e5-d7eda0013a20 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.836424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66b377-c1e4-431e-81e5-d7eda0013a20 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.836717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66b377-c1e4-431e-81e5-d7eda0013a20 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:42.841811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 372/738] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:42.841811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66b377-c1e4-431e-81e5-d7eda0013a20 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:42.869446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b4840-8f7d-4b6c-82b9-cf1fcf8025b9 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:42.905658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d140297-6157-4b7e-99e3-b61296bb29d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:42.906262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d140297-6157-4b7e-99e3-b61296bb29d5 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.906557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d140297-6157-4b7e-99e3-b61296bb29d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.906801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d140297-6157-4b7e-99e3-b61296bb29d5 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:42.907448 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d140297-6157-4b7e-99e3-b61296bb29d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:42.907774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d140297-6157-4b7e-99e3-b61296bb29d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:42.909861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d140297-6157-4b7e-99e3-b61296bb29d5 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:42.924709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 367/739] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:42.926307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f352-4c6e-4d27-b170-7adf562cc2c2 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.926590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f352-4c6e-4d27-b170-7adf562cc2c2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.926864 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f352-4c6e-4d27-b170-7adf562cc2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:42.927797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:42.946779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6af6f352-4c6e-4d27-b170-7adf562cc2c2 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:42.947968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6af6f352-4c6e-4d27-b170-7adf562cc2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:42.951120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66b377-c1e4-431e-81e5-d7eda0013a20 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:42.988866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0acd7-0b83-4294-92ef-b641d91ca103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:42.989491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0acd7-0b83-4294-92ef-b641d91ca103 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:42.989765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0acd7-0b83-4294-92ef-b641d91ca103 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:42.990029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0acd7-0b83-4294-92ef-b641d91ca103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.000916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0acd7-0b83-4294-92ef-b641d91ca103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:43.002748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0acd7-0b83-4294-92ef-b641d91ca103 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:43.024536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 368/740] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-2fc0acd7-0b83-4294-92ef-b641d91ca103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:43.025050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:43.026141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:43.027163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:43.027163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:43.027163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:43.027328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb02324-dd96-4c31-ab18-256cceb6cc7f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:43.027328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb02324-dd96-4c31-ab18-256cceb6cc7f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:43.027328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb02324-dd96-4c31-ab18-256cceb6cc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.027513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 373/741] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:43.027513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0acd7-0b83-4294-92ef-b641d91ca103 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:43.049690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ccb02324-dd96-4c31-ab18-256cceb6cc7f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:43.051091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ccb02324-dd96-4c31-ab18-256cceb6cc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:43.066337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:43.133082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 369/742] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 00:17:43.133954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb02324-dd96-4c31-ab18-256cceb6cc7f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:43.173638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0acd7-0b83-4294-92ef-b641d91ca103 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:43.232976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:43.246000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b98363-7f18-4acc-9a6b-b66ffbf85834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:43.246649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b98363-7f18-4acc-9a6b-b66ffbf85834 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:43.246928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b98363-7f18-4acc-9a6b-b66ffbf85834 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:43.247200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b98363-7f18-4acc-9a6b-b66ffbf85834 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.260516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b98363-7f18-4acc-9a6b-b66ffbf85834 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:43.260516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b98363-7f18-4acc-9a6b-b66ffbf85834 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-b6b98363-7f18-4acc-9a6b-b66ffbf85834 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:43.287129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:43.288130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:43.289107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 374/743] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:17:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:43.295728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b98363-7f18-4acc-9a6b-b66ffbf85834 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:43.337883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb02324-dd96-4c31-ab18-256cceb6cc7f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:43.385161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c645da-eba9-40a7-92cd-88e6dfd07d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:43.385766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c645da-eba9-40a7-92cd-88e6dfd07d65 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:43.386042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c645da-eba9-40a7-92cd-88e6dfd07d65 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:43.386309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c645da-eba9-40a7-92cd-88e6dfd07d65 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.386949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c645da-eba9-40a7-92cd-88e6dfd07d65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:43.387249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c645da-eba9-40a7-92cd-88e6dfd07d65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:43.388648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c645da-eba9-40a7-92cd-88e6dfd07d65 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:43.401270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 370/744] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:43.402085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c645da-eba9-40a7-92cd-88e6dfd07d65 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:43.416901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:43.436989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b98363-7f18-4acc-9a6b-b66ffbf85834 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:43.478165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8b4a8-6e3f-4b8e-8dbc-80c51e2205ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:43.478846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8b4a8-6e3f-4b8e-8dbc-80c51e2205ee None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:43.479036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8b4a8-6e3f-4b8e-8dbc-80c51e2205ee None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:43.479304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8b4a8-6e3f-4b8e-8dbc-80c51e2205ee None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.480064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8b4a8-6e3f-4b8e-8dbc-80c51e2205ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:43.480318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8b4a8-6e3f-4b8e-8dbc-80c51e2205ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:43.481799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8b4a8-6e3f-4b8e-8dbc-80c51e2205ee None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:43.503957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 375/745] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 00:17:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:43.503957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8b4a8-6e3f-4b8e-8dbc-80c51e2205ee None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:43.513999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c645da-eba9-40a7-92cd-88e6dfd07d65 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:43.550686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcd34e-1f34-4291-b82c-25f1117e3dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:43.551314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcd34e-1f34-4291-b82c-25f1117e3dc7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:43.551591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcd34e-1f34-4291-b82c-25f1117e3dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:43.551867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcd34e-1f34-4291-b82c-25f1117e3dc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.556580 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:43.567201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcd34e-1f34-4291-b82c-25f1117e3dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-e0bcd34e-1f34-4291-b82c-25f1117e3dc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:43.573292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:43.574236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:43.575128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 371/746] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:43.575835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcd34e-1f34-4291-b82c-25f1117e3dc7 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:43.620543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8b4a8-6e3f-4b8e-8dbc-80c51e2205ee None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:43.646470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:43.666090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03aa5bf-ac0c-4ed8-8ede-f23968b67fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:43.666722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03aa5bf-ac0c-4ed8-8ede-f23968b67fb7 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:43.667001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03aa5bf-ac0c-4ed8-8ede-f23968b67fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:43.667274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03aa5bf-ac0c-4ed8-8ede-f23968b67fb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.675016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03aa5bf-ac0c-4ed8-8ede-f23968b67fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:43.676472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03aa5bf-ac0c-4ed8-8ede-f23968b67fb7 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:43.683011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcd34e-1f34-4291-b82c-25f1117e3dc7 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-e03aa5bf-ac0c-4ed8-8ede-f23968b67fb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:43.687890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:43.688773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:43.690206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:43.690206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:43.690206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:43.690206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:43.690206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:43.690206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:43.690206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:43.690206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:43.690206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 376/747] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:43.695012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03aa5bf-ac0c-4ed8-8ede-f23968b67fb7 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:43.720013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18fc2e-a5bf-40ac-899c-fd78f87db4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:43.720671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18fc2e-a5bf-40ac-899c-fd78f87db4c3 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:43.721474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18fc2e-a5bf-40ac-899c-fd78f87db4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:43.721744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18fc2e-a5bf-40ac-899c-fd78f87db4c3 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.722411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18fc2e-a5bf-40ac-899c-fd78f87db4c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:43.723828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18fc2e-a5bf-40ac-899c-fd78f87db4c3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:43.740628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 372/748] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:17:43 2022] POST /identity/v3/services => generated 240 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:43.741891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18fc2e-a5bf-40ac-899c-fd78f87db4c3 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:43.839037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03aa5bf-ac0c-4ed8-8ede-f23968b67fb7 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:43.856557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:43.875052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18fc2e-a5bf-40ac-899c-fd78f87db4c3 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:43.892626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9582ef0-92f9-4305-a125-7f8cbcc60ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:43.893811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9582ef0-92f9-4305-a125-7f8cbcc60ebd None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:43.894270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9582ef0-92f9-4305-a125-7f8cbcc60ebd None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:43.894863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9582ef0-92f9-4305-a125-7f8cbcc60ebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.902266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:43.917791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba2e34-3165-4047-9f93-c2406e284cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:43.918398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba2e34-3165-4047-9f93-c2406e284cac None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:43.918674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba2e34-3165-4047-9f93-c2406e284cac None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:43.919180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba2e34-3165-4047-9f93-c2406e284cac None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:43.919604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba2e34-3165-4047-9f93-c2406e284cac None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:43.919899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba2e34-3165-4047-9f93-c2406e284cac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:43.921319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bba2e34-3165-4047-9f93-c2406e284cac None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:43.932042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 373/749] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:43.932963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba2e34-3165-4047-9f93-c2406e284cac None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:43.984236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9582ef0-92f9-4305-a125-7f8cbcc60ebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:43.987522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9582ef0-92f9-4305-a125-7f8cbcc60ebd None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:44.060957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba2e34-3165-4047-9f93-c2406e284cac None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:44.110442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a92bf-fc9f-43b1-a3d1-f042317eecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:44.116099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 377/750] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:17:44.116099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9582ef0-92f9-4305-a125-7f8cbcc60ebd None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:44.116816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a92bf-fc9f-43b1-a3d1-f042317eecee None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:44.117262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a92bf-fc9f-43b1-a3d1-f042317eecee None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:44.118611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a92bf-fc9f-43b1-a3d1-f042317eecee None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:44.119488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a92bf-fc9f-43b1-a3d1-f042317eecee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:44.119944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a92bf-fc9f-43b1-a3d1-f042317eecee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:44.121648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a92bf-fc9f-43b1-a3d1-f042317eecee None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:44.127027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:44.141782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 374/751] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:44.144217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a92bf-fc9f-43b1-a3d1-f042317eecee None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:44.205148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:44.227247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9582ef0-92f9-4305-a125-7f8cbcc60ebd None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:44.274078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2240-d976-425c-bd9f-51bcef11ee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:44.274719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2240-d976-425c-bd9f-51bcef11ee65 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:44.275000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2240-d976-425c-bd9f-51bcef11ee65 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:44.275271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb2240-d976-425c-bd9f-51bcef11ee65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:44.282810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a92bf-fc9f-43b1-a3d1-f042317eecee None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:44.288716 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb2240-d976-425c-bd9f-51bcef11ee65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:44.290180 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb2240-d976-425c-bd9f-51bcef11ee65 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-2bdb2240-d976-425c-bd9f-51bcef11ee65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:44.310124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:44.311083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:44.311995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:44.311995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:44.311995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:44.316263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 378/752] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:44.317483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2240-d976-425c-bd9f-51bcef11ee65 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:44.325827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f1fe2-8f27-4e75-8a2a-712fa17eddff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:44.326813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f1fe2-8f27-4e75-8a2a-712fa17eddff None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:44.327283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f1fe2-8f27-4e75-8a2a-712fa17eddff None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:44.327788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f1fe2-8f27-4e75-8a2a-712fa17eddff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:44.339644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f1fe2-8f27-4e75-8a2a-712fa17eddff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:44.341546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f1fe2-8f27-4e75-8a2a-712fa17eddff None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-5d8f1fe2-8f27-4e75-8a2a-712fa17eddff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:44.358735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:44.359678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:44.360618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:44.360618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:44.360618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:44.362749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 375/753] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:44.363857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f1fe2-8f27-4e75-8a2a-712fa17eddff None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:44.391518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:44.435560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb2240-d976-425c-bd9f-51bcef11ee65 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:44.474416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4da112-2769-443a-8659-dc479bc542ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:44.475698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4da112-2769-443a-8659-dc479bc542ec None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:44.476265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4da112-2769-443a-8659-dc479bc542ec None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:44.476774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4da112-2769-443a-8659-dc479bc542ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:44.477756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4da112-2769-443a-8659-dc479bc542ec None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=ed93469862874c75951d95e28929c904, user.id=43d6c2a533e849cab240257cc1450c9f) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:44.478293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4da112-2769-443a-8659-dc479bc542ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:44.480015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4da112-2769-443a-8659-dc479bc542ec None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:44.486205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 379/754] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:44 2022] GET /identity/v3/role_assignments?user.id=43d6c2a533e849cab240257cc1450c9f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:44.487581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4da112-2769-443a-8659-dc479bc542ec None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:44.488733 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:44.503758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f1fe2-8f27-4e75-8a2a-712fa17eddff None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:44.604450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064dfc5-7972-4902-a8b4-f8448321e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:44.605096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064dfc5-7972-4902-a8b4-f8448321e0e3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:44.605486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064dfc5-7972-4902-a8b4-f8448321e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:44.605754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064dfc5-7972-4902-a8b4-f8448321e0e3 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:44.606536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064dfc5-7972-4902-a8b4-f8448321e0e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:44.606837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064dfc5-7972-4902-a8b4-f8448321e0e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:44.608242 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064dfc5-7972-4902-a8b4-f8448321e0e3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:44.623082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 376/755] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:44.626744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4da112-2769-443a-8659-dc479bc542ec None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:44.629075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f21dba-1182-4583-86f2-48a473cbdc81 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:44.629361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f21dba-1182-4583-86f2-48a473cbdc81 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:44.629628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f21dba-1182-4583-86f2-48a473cbdc81 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:44.631095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 377/756] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:44.631833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f21dba-1182-4583-86f2-48a473cbdc81 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:44.655567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:44.668765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ba0dd-ce8d-4f59-9d6e-f031329e7b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:44.669826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ba0dd-ce8d-4f59-9d6e-f031329e7b7a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:44.670373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ba0dd-ce8d-4f59-9d6e-f031329e7b7a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:44.670885 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ba0dd-ce8d-4f59-9d6e-f031329e7b7a None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:44.671713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ba0dd-ce8d-4f59-9d6e-f031329e7b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:44.672318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ba0dd-ce8d-4f59-9d6e-f031329e7b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:44.674785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ba0dd-ce8d-4f59-9d6e-f031329e7b7a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:44.690304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 380/757] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:44.692044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6772a9-2f2c-4a05-a4db-1397c1f8c09f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:44.692488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6772a9-2f2c-4a05-a4db-1397c1f8c09f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:44.692944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6772a9-2f2c-4a05-a4db-1397c1f8c09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:44.715605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-df6772a9-2f2c-4a05-a4db-1397c1f8c09f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:44.718215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-df6772a9-2f2c-4a05-a4db-1397c1f8c09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:44.751999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f21dba-1182-4583-86f2-48a473cbdc81 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:44.796189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb7b38-7855-4a23-8674-b082e9796df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:44.797041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb7b38-7855-4a23-8674-b082e9796df2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:44.797421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb7b38-7855-4a23-8674-b082e9796df2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:44.797857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb7b38-7855-4a23-8674-b082e9796df2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:44.807228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb7b38-7855-4a23-8674-b082e9796df2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:44.808883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb7b38-7855-4a23-8674-b082e9796df2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:44.810082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 381/758] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:44.811148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6772a9-2f2c-4a05-a4db-1397c1f8c09f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f7cb7b38-7855-4a23-8674-b082e9796df2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:44.822672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:44.823842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:44.825018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:44.826872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 378/759] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 00:17:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:44.837656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb7b38-7855-4a23-8674-b082e9796df2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:44.880855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:44.964439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6772a9-2f2c-4a05-a4db-1397c1f8c09f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:45.000856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:45.037581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc8f7a-7a97-41dd-8d95-27d9d6e8202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:45.038384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc8f7a-7a97-41dd-8d95-27d9d6e8202f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.039590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc8f7a-7a97-41dd-8d95-27d9d6e8202f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.039960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc8f7a-7a97-41dd-8d95-27d9d6e8202f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.054162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc8f7a-7a97-41dd-8d95-27d9d6e8202f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:45.055754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc8f7a-7a97-41dd-8d95-27d9d6e8202f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:45.076187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb7b38-7855-4a23-8674-b082e9796df2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-6ddc8f7a-7a97-41dd-8d95-27d9d6e8202f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:45.082985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:45.084038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:45.085195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:45.085195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:45.085195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:45.085195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:45.085195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:45.085195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:45.085195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:45.085195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:45.086829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 382/760] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:45.093796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc8f7a-7a97-41dd-8d95-27d9d6e8202f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:45.117456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a462274-f066-4981-907c-e6f2a3d229fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:45.118331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a462274-f066-4981-907c-e6f2a3d229fa None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.118751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a462274-f066-4981-907c-e6f2a3d229fa None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.119162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a462274-f066-4981-907c-e6f2a3d229fa None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.120628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a462274-f066-4981-907c-e6f2a3d229fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:45.123636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a462274-f066-4981-907c-e6f2a3d229fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:45.125268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a462274-f066-4981-907c-e6f2a3d229fa None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:45.145173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 379/761] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 00:17:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:45.151850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a462274-f066-4981-907c-e6f2a3d229fa None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:45.259516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc8f7a-7a97-41dd-8d95-27d9d6e8202f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:45.267871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:45.303879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a462274-f066-4981-907c-e6f2a3d229fa None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:45.308675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887f120-248b-45db-9f6f-a47c88bd97be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:45.309391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887f120-248b-45db-9f6f-a47c88bd97be None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.309836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887f120-248b-45db-9f6f-a47c88bd97be None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.310207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887f120-248b-45db-9f6f-a47c88bd97be None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.311003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887f120-248b-45db-9f6f-a47c88bd97be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:45.311409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887f120-248b-45db-9f6f-a47c88bd97be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:45.312985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887f120-248b-45db-9f6f-a47c88bd97be None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:45.331287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 383/762] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:45.332330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887f120-248b-45db-9f6f-a47c88bd97be None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:45.353054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc40dc1-71ec-4912-9e1d-846af26aa63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:45.354344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc40dc1-71ec-4912-9e1d-846af26aa63e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.354763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc40dc1-71ec-4912-9e1d-846af26aa63e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.355198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc40dc1-71ec-4912-9e1d-846af26aa63e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.371197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:45.372422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc40dc1-71ec-4912-9e1d-846af26aa63e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:45.374431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc40dc1-71ec-4912-9e1d-846af26aa63e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-9fc40dc1-71ec-4912-9e1d-846af26aa63e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:45.396986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:45.398142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:45.399230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:45.399230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:45.399230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:45.404799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 380/763] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:45.405775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc40dc1-71ec-4912-9e1d-846af26aa63e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:45.461473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887f120-248b-45db-9f6f-a47c88bd97be None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:45.505908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54df4c-4c82-404f-9d07-31ab16012184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:45.506712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54df4c-4c82-404f-9d07-31ab16012184 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.507058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54df4c-4c82-404f-9d07-31ab16012184 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.507397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54df4c-4c82-404f-9d07-31ab16012184 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.508067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54df4c-4c82-404f-9d07-31ab16012184 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:45.509543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54df4c-4c82-404f-9d07-31ab16012184 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-7a54df4c-4c82-404f-9d07-31ab16012184 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:45.519285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:45.520261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 00:17:45.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:45.523985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc40dc1-71ec-4912-9e1d-846af26aa63e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:45.524863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 384/764] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 1 00:17:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:45.525717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54df4c-4c82-404f-9d07-31ab16012184 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:45.560262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:45.570061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ddb55-adc4-437a-a9a1-61258b77b62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:45.570895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ddb55-adc4-437a-a9a1-61258b77b62b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.571319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ddb55-adc4-437a-a9a1-61258b77b62b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.571766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ddb55-adc4-437a-a9a1-61258b77b62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.572779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ddb55-adc4-437a-a9a1-61258b77b62b None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=6eb1b515c4a74ba5a9da1c94ec914a75) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:45.573234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ddb55-adc4-437a-a9a1-61258b77b62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:45.574914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ddb55-adc4-437a-a9a1-61258b77b62b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:45.581984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 381/765] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:45 2022] GET /identity/v3/role_assignments?user.id=6eb1b515c4a74ba5a9da1c94ec914a75&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:45.583009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ddb55-adc4-437a-a9a1-61258b77b62b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:45.701013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ddb55-adc4-437a-a9a1-61258b77b62b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:45.701746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:45.708078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54df4c-4c82-404f-9d07-31ab16012184 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:45.743629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdba6b-dbb7-4e40-a17b-1f6f94665065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:45.744694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdba6b-dbb7-4e40-a17b-1f6f94665065 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.745180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdba6b-dbb7-4e40-a17b-1f6f94665065 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.745656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdba6b-dbb7-4e40-a17b-1f6f94665065 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.746513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdba6b-dbb7-4e40-a17b-1f6f94665065 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:45.747021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdba6b-dbb7-4e40-a17b-1f6f94665065 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:45.748689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdba6b-dbb7-4e40-a17b-1f6f94665065 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:45.754698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 382/766] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 00:17:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:45.760418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdba6b-dbb7-4e40-a17b-1f6f94665065 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:45.801321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023489e-2b44-4740-b6be-1db15c6cd313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:45.801941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023489e-2b44-4740-b6be-1db15c6cd313 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.802221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023489e-2b44-4740-b6be-1db15c6cd313 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.802554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023489e-2b44-4740-b6be-1db15c6cd313 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.803213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023489e-2b44-4740-b6be-1db15c6cd313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:45.803517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023489e-2b44-4740-b6be-1db15c6cd313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:45.806000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023489e-2b44-4740-b6be-1db15c6cd313 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:45.822938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 385/767] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:45.828964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023489e-2b44-4740-b6be-1db15c6cd313 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:45.831826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:45.881789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdba6b-dbb7-4e40-a17b-1f6f94665065 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:45.918234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a6c29-2f37-4182-9b61-b881e845ad0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:45.919318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a6c29-2f37-4182-9b61-b881e845ad0e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.919809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a6c29-2f37-4182-9b61-b881e845ad0e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.920332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a6c29-2f37-4182-9b61-b881e845ad0e None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.921372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a6c29-2f37-4182-9b61-b881e845ad0e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:45.921879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a6c29-2f37-4182-9b61-b881e845ad0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:45.923484 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a6c29-2f37-4182-9b61-b881e845ad0e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:45.930934 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 383/768] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 00:17:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:45.932715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c1df-02e6-4ca1-bc80-0d4f96c1e045 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:45.933212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c1df-02e6-4ca1-bc80-0d4f96c1e045 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:45.933683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c1df-02e6-4ca1-bc80-0d4f96c1e045 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:45.935384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 384/769] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:45.937186 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875c1df-02e6-4ca1-bc80-0d4f96c1e045 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:45.960888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023489e-2b44-4740-b6be-1db15c6cd313 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:46.007978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33253161-ff6c-43cf-980e-fef2fad86e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:46.008637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33253161-ff6c-43cf-980e-fef2fad86e20 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.008947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33253161-ff6c-43cf-980e-fef2fad86e20 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.009224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33253161-ff6c-43cf-980e-fef2fad86e20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.018718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33253161-ff6c-43cf-980e-fef2fad86e20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:46.021060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33253161-ff6c-43cf-980e-fef2fad86e20 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-33253161-ff6c-43cf-980e-fef2fad86e20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:46.035789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:46.036765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:46.037639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:46.037639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:46.037639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:46.037639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:46.037639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:46.037639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:46.037639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:46.037639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:46.038078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 386/770] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:46.043542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec02578-3b61-4d2a-b903-5dfc950ae93e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.043542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec02578-3b61-4d2a-b903-5dfc950ae93e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.043542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec02578-3b61-4d2a-b903-5dfc950ae93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.062878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875c1df-02e6-4ca1-bc80-0d4f96c1e045 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:46.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-6ec02578-3b61-4d2a-b903-5dfc950ae93e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:46.069634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6ec02578-3b61-4d2a-b903-5dfc950ae93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:46.086606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:46.092393 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:46.106665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ccd24-43e2-434a-94bb-7d33df7f9444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:46.107882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ccd24-43e2-434a-94bb-7d33df7f9444 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.108872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ccd24-43e2-434a-94bb-7d33df7f9444 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.109635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ccd24-43e2-434a-94bb-7d33df7f9444 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.110811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ccd24-43e2-434a-94bb-7d33df7f9444 None admin] RBAC: Adding query filter params (interface=public, service_id=683c0badac644f839aea98dac4e2c37b, region_id=RegionOne) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:46.111811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ccd24-43e2-434a-94bb-7d33df7f9444 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:46.113713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ccd24-43e2-434a-94bb-7d33df7f9444 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:46.124376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 385/771] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 00:17:45 2022] GET /identity/v3/endpoints?service_id=683c0badac644f839aea98dac4e2c37b&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:46.125529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ccd24-43e2-434a-94bb-7d33df7f9444 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:46.177540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 387/772] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:46.179058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec02578-3b61-4d2a-b903-5dfc950ae93e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:46.259411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ccd24-43e2-434a-94bb-7d33df7f9444 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:46.298387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af4508-d3fd-4518-bd35-7664951b4197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:46.299731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af4508-d3fd-4518-bd35-7664951b4197 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.300130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af4508-d3fd-4518-bd35-7664951b4197 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.300520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af4508-d3fd-4518-bd35-7664951b4197 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.301315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af4508-d3fd-4518-bd35-7664951b4197 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:46.301722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af4508-d3fd-4518-bd35-7664951b4197 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:46.303254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af4508-d3fd-4518-bd35-7664951b4197 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:46.311649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec02578-3b61-4d2a-b903-5dfc950ae93e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:46.316430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 386/773] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:46.322473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af4508-d3fd-4518-bd35-7664951b4197 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:46.376363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63d3b4-598a-44da-8e88-a390f50b908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:46.378146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63d3b4-598a-44da-8e88-a390f50b908a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.378429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63d3b4-598a-44da-8e88-a390f50b908a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.378707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63d3b4-598a-44da-8e88-a390f50b908a None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.379326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63d3b4-598a-44da-8e88-a390f50b908a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:46.384259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63d3b4-598a-44da-8e88-a390f50b908a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:46.395432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 388/774] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 00:17:46 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:46.409518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:46.469980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af4508-d3fd-4518-bd35-7664951b4197 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:46.509007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7e885-0ce0-4872-a2de-febbcbf3fea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:46.509654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7e885-0ce0-4872-a2de-febbcbf3fea5 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.509935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7e885-0ce0-4872-a2de-febbcbf3fea5 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.510265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7e885-0ce0-4872-a2de-febbcbf3fea5 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/fb204040ab2f4168b35c3e78bacb849f/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.512358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc7e885-0ce0-4872-a2de-febbcbf3fea5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=fb204040ab2f4168b35c3e78bacb849f, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:46.513854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc7e885-0ce0-4872-a2de-febbcbf3fea5 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:46.531100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 387/775] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:17:46 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/fb204040ab2f4168b35c3e78bacb849f/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:46.550157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63d3b4-598a-44da-8e88-a390f50b908a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:46.577108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:46.685042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63d3b4-598a-44da-8e88-a390f50b908a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:46.725359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c6dd-ddc0-43d5-a2f1-878259f2ee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:46.725990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c6dd-ddc0-43d5-a2f1-878259f2ee33 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.726266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c6dd-ddc0-43d5-a2f1-878259f2ee33 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.726533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49c6dd-ddc0-43d5-a2f1-878259f2ee33 None admin] PATH_INFO: `/v3/services/network` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.727115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49c6dd-ddc0-43d5-a2f1-878259f2ee33 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:46.728483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49c6dd-ddc0-43d5-a2f1-878259f2ee33 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:46.739183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5637f10-f550-4d81-9d29-40d70b8a4470 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.739494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5637f10-f550-4d81-9d29-40d70b8a4470 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.739725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5637f10-f550-4d81-9d29-40d70b8a4470 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-4a49c6dd-ddc0-43d5-a2f1-878259f2ee33 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:46.740848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:46.741793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 00:17:46.742613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:46.743340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 388/776] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:46.745070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 389/777] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:46 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:46.746231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3db65-c6d1-4c32-9c4a-f447582dc7d4 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.746539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3db65-c6d1-4c32-9c4a-f447582dc7d4 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.746783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3db65-c6d1-4c32-9c4a-f447582dc7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.750484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c6dd-ddc0-43d5-a2f1-878259f2ee33 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:46.783110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e6d3db65-c6d1-4c32-9c4a-f447582dc7d4 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:46.786147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e6d3db65-c6d1-4c32-9c4a-f447582dc7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:46.879522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:46.906132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 389/778] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:46.916928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49c6dd-ddc0-43d5-a2f1-878259f2ee33 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:46.952998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870e820-a40e-4f91-a223-03ce79efaf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:46.953601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870e820-a40e-4f91-a223-03ce79efaf73 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:46.953874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870e820-a40e-4f91-a223-03ce79efaf73 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:46.954140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870e820-a40e-4f91-a223-03ce79efaf73 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:46.954765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870e820-a40e-4f91-a223-03ce79efaf73 None admin] RBAC: Adding query filter params (name=network) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:46.955063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870e820-a40e-4f91-a223-03ce79efaf73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:46.957057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870e820-a40e-4f91-a223-03ce79efaf73 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:46.965822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 390/779] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:17:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:46.967944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3db65-c6d1-4c32-9c4a-f447582dc7d4 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:47.028686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:47.115645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3db65-c6d1-4c32-9c4a-f447582dc7d4 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:47.151796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26936cab-7d86-46f4-b78c-c9dc4c0ea155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:47.152799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26936cab-7d86-46f4-b78c-c9dc4c0ea155 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:47.153280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26936cab-7d86-46f4-b78c-c9dc4c0ea155 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:47.153787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26936cab-7d86-46f4-b78c-c9dc4c0ea155 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:47.154631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26936cab-7d86-46f4-b78c-c9dc4c0ea155 None admin] RBAC: Adding query filter params (type=network) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:47.155132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26936cab-7d86-46f4-b78c-c9dc4c0ea155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:47.156742 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26936cab-7d86-46f4-b78c-c9dc4c0ea155 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:47.162810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 390/780] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:17:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:47.168798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870e820-a40e-4f91-a223-03ce79efaf73 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:47.257517 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26936cab-7d86-46f4-b78c-c9dc4c0ea155 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:47.317545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870e820-a40e-4f91-a223-03ce79efaf73 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:47.412445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:47.416243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26936cab-7d86-46f4-b78c-c9dc4c0ea155 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:47.418638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70593d-eaed-4da1-a830-da4f84d17c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:47.419267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70593d-eaed-4da1-a830-da4f84d17c4e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:47.419549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70593d-eaed-4da1-a830-da4f84d17c4e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:47.419905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70593d-eaed-4da1-a830-da4f84d17c4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:47.420640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70593d-eaed-4da1-a830-da4f84d17c4e None admin] RBAC: Adding query filter params (interface=public, service_id=192e77a759e64d9b9e26b3ae570a34a2, region_id=RegionOne) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:47.420951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70593d-eaed-4da1-a830-da4f84d17c4e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:47.422325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70593d-eaed-4da1-a830-da4f84d17c4e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:47.446444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 391/781] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 00:17:47 2022] GET /identity/v3/endpoints?service_id=192e77a759e64d9b9e26b3ae570a34a2&interface=public®ion_id=RegionOne => generated 194 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:47.457601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70593d-eaed-4da1-a830-da4f84d17c4e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:47.467621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39b232-fd20-481f-8408-d611b13ea19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:47.468360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39b232-fd20-481f-8408-d611b13ea19d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:47.468708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39b232-fd20-481f-8408-d611b13ea19d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:47.469203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:47.469311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39b232-fd20-481f-8408-d611b13ea19d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:47.496929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39b232-fd20-481f-8408-d611b13ea19d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-6f39b232-fd20-481f-8408-d611b13ea19d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:47.501819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:47.503373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:47.504196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:47.513422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 391/782] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:47.516938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39b232-fd20-481f-8408-d611b13ea19d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:47.646905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70593d-eaed-4da1-a830-da4f84d17c4e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:47.696559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39b232-fd20-481f-8408-d611b13ea19d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:47.716263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ba36d-8fbc-4df6-94fb-55541f28a3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:47.716903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ba36d-8fbc-4df6-94fb-55541f28a3f3 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:47.717180 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ba36d-8fbc-4df6-94fb-55541f28a3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:47.717449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ba36d-8fbc-4df6-94fb-55541f28a3f3 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:47.718056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ba36d-8fbc-4df6-94fb-55541f28a3f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:47.719435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ba36d-8fbc-4df6-94fb-55541f28a3f3 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:47.729664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 392/783] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 00:17:47 2022] GET /identity/v3/services => generated 772 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:47.760306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d77c78-589b-4e96-871b-e7ba2709eaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:47.760953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d77c78-589b-4e96-871b-e7ba2709eaee None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:47.761226 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d77c78-589b-4e96-871b-e7ba2709eaee None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:47.761494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d77c78-589b-4e96-871b-e7ba2709eaee None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:47.762163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d77c78-589b-4e96-871b-e7ba2709eaee None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:47.762461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d77c78-589b-4e96-871b-e7ba2709eaee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:47.764072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d77c78-589b-4e96-871b-e7ba2709eaee None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:47.764670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c3d3a-45b0-4e18-979f-5e74588f5608 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:47.764956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c3d3a-45b0-4e18-979f-5e74588f5608 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:47.765233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c3d3a-45b0-4e18-979f-5e74588f5608 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:47.766767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 393/784] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:17:47.773035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f2cec-a20f-4721-b3df-e377db31242b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:47.773336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f2cec-a20f-4721-b3df-e377db31242b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:47.773598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f2cec-a20f-4721-b3df-e377db31242b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:47.781732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 392/785] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:47.787872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d77c78-589b-4e96-871b-e7ba2709eaee None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:47.795133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ac7f2cec-a20f-4721-b3df-e377db31242b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:47.796401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ac7f2cec-a20f-4721-b3df-e377db31242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:47.829862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:47.891024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:47.893271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 394/786] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:17:47.964403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d77c78-589b-4e96-871b-e7ba2709eaee None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:48.017286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44776bba-aa5b-4680-8976-6c5314e68a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:48.018459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44776bba-aa5b-4680-8976-6c5314e68a8f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.018796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44776bba-aa5b-4680-8976-6c5314e68a8f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.019074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44776bba-aa5b-4680-8976-6c5314e68a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.030223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44776bba-aa5b-4680-8976-6c5314e68a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:48.031731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44776bba-aa5b-4680-8976-6c5314e68a8f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-44776bba-aa5b-4680-8976-6c5314e68a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:48.046716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:48.047644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:48.048583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:48.048583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:48.048583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:48.048583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 393/787] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:48.055961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f2cec-a20f-4721-b3df-e377db31242b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:48.165446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f2cec-a20f-4721-b3df-e377db31242b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:48.179520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44776bba-aa5b-4680-8976-6c5314e68a8f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:48.206923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a12f3-1d3c-4cf6-a50c-7d9b9f7427ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:48.207592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a12f3-1d3c-4cf6-a50c-7d9b9f7427ac None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.207816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a12f3-1d3c-4cf6-a50c-7d9b9f7427ac None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.208093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a12f3-1d3c-4cf6-a50c-7d9b9f7427ac None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.208836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a12f3-1d3c-4cf6-a50c-7d9b9f7427ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:48.209141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a12f3-1d3c-4cf6-a50c-7d9b9f7427ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:48.211515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a12f3-1d3c-4cf6-a50c-7d9b9f7427ac None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:48.230843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 395/788] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:48.237283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a12f3-1d3c-4cf6-a50c-7d9b9f7427ac None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:48.295272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:48.329370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44776bba-aa5b-4680-8976-6c5314e68a8f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:48.370522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaa5bf-1e14-4893-a243-d10ed116558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:48.371155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaa5bf-1e14-4893-a243-d10ed116558b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.371429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaa5bf-1e14-4893-a243-d10ed116558b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.371699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdaa5bf-1e14-4893-a243-d10ed116558b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.397897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdaa5bf-1e14-4893-a243-d10ed116558b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-3bdaa5bf-1e14-4893-a243-d10ed116558b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:48.404827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:48.405789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:48.406613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:48.407790 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:48.415380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 394/789] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:48.416284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaa5bf-1e14-4893-a243-d10ed116558b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:48.449006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a12f3-1d3c-4cf6-a50c-7d9b9f7427ac None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:48.526252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f488c-8cd5-4419-b0b3-3131f493c30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:48.526808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f488c-8cd5-4419-b0b3-3131f493c30b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.527086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f488c-8cd5-4419-b0b3-3131f493c30b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.527355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f488c-8cd5-4419-b0b3-3131f493c30b None admin] PATH_INFO: `/v3/users/swift` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.529990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdaa5bf-1e14-4893-a243-d10ed116558b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:48.540758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f488c-8cd5-4419-b0b3-3131f493c30b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:48.542219 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f488c-8cd5-4419-b0b3-3131f493c30b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:48.561427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:48.570198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad580ad0-f202-42af-817b-1c7486829385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:48.571083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad580ad0-f202-42af-817b-1c7486829385 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.571502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad580ad0-f202-42af-817b-1c7486829385 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.571916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad580ad0-f202-42af-817b-1c7486829385 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.572791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad580ad0-f202-42af-817b-1c7486829385 None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:48.573232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad580ad0-f202-42af-817b-1c7486829385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-c94f488c-8cd5-4419-b0b3-3131f493c30b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:48.573575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:48.574658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:48.575829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:48.577090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 396/790] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:48.577858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad580ad0-f202-42af-817b-1c7486829385 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:48.585498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f488c-8cd5-4419-b0b3-3131f493c30b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:48.588134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 395/791] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:48.593869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad580ad0-f202-42af-817b-1c7486829385 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:48.703290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad580ad0-f202-42af-817b-1c7486829385 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:48.743694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd51b40-6f62-4738-8805-477973ed8950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:48.745193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd51b40-6f62-4738-8805-477973ed8950 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.745761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd51b40-6f62-4738-8805-477973ed8950 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.748389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd51b40-6f62-4738-8805-477973ed8950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.768444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd51b40-6f62-4738-8805-477973ed8950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:48.770150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd51b40-6f62-4738-8805-477973ed8950 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-2bd51b40-6f62-4738-8805-477973ed8950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:48.785401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:48.786512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:48.787478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:48.787478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:48.787478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:48.789208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 396/792] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:48.794837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44449c-eb45-4cad-9c9a-b2ff09e74d70 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.794973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f488c-8cd5-4419-b0b3-3131f493c30b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:48.795423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44449c-eb45-4cad-9c9a-b2ff09e74d70 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.795845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44449c-eb45-4cad-9c9a-b2ff09e74d70 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.797331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 397/793] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:48.798222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44449c-eb45-4cad-9c9a-b2ff09e74d70 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:48.818449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:48.836289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b8790-1654-4e1f-aa41-4c965e3f62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:48.837227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b8790-1654-4e1f-aa41-4c965e3f62e6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.837671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b8790-1654-4e1f-aa41-4c965e3f62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.838098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b8790-1654-4e1f-aa41-4c965e3f62e6 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.838964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b8790-1654-4e1f-aa41-4c965e3f62e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:48.839416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b8790-1654-4e1f-aa41-4c965e3f62e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:48.841020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b8790-1654-4e1f-aa41-4c965e3f62e6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:48.858056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:48.867257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 397/794] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 00:17:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:48.868826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2cf03-bba4-4951-8166-aee0ffdbdda7 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.869249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2cf03-bba4-4951-8166-aee0ffdbdda7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.869679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2cf03-bba4-4951-8166-aee0ffdbdda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.892203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-a1f2cf03-bba4-4951-8166-aee0ffdbdda7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:48.893859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a1f2cf03-bba4-4951-8166-aee0ffdbdda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:48.924138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44449c-eb45-4cad-9c9a-b2ff09e74d70 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:48.961922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9794b-65fe-4e1d-bd92-0dfa462b7a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:48.962577 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9794b-65fe-4e1d-bd92-0dfa462b7a1d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:48.962858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9794b-65fe-4e1d-bd92-0dfa462b7a1d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:48.963120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9794b-65fe-4e1d-bd92-0dfa462b7a1d None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:48.963767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9794b-65fe-4e1d-bd92-0dfa462b7a1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:48.964066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9794b-65fe-4e1d-bd92-0dfa462b7a1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:48.965500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9794b-65fe-4e1d-bd92-0dfa462b7a1d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:48.982698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 398/795] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:48.982698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9794b-65fe-4e1d-bd92-0dfa462b7a1d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:48.985524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 398/796] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:48.986825 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2cf03-bba4-4951-8166-aee0ffdbdda7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.097367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9794b-65fe-4e1d-bd92-0dfa462b7a1d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:49.122204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2cf03-bba4-4951-8166-aee0ffdbdda7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:49.124839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:49.134836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea74a7-fc2e-4d95-9e2f-4635e2864fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:49.135623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea74a7-fc2e-4d95-9e2f-4635e2864fed None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.136006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea74a7-fc2e-4d95-9e2f-4635e2864fed None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.136377 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea74a7-fc2e-4d95-9e2f-4635e2864fed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.137139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:49.150030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea74a7-fc2e-4d95-9e2f-4635e2864fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:49.151872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea74a7-fc2e-4d95-9e2f-4635e2864fed None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-ccea74a7-fc2e-4d95-9e2f-4635e2864fed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:49.167756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:49.168940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:49.170007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:49.170007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:49.170007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:49.170470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c1d49-8fa3-44fa-8b40-defecebe159d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:49.171344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c1d49-8fa3-44fa-8b40-defecebe159d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.171756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c1d49-8fa3-44fa-8b40-defecebe159d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.171933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 399/797] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:49.172306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c1d49-8fa3-44fa-8b40-defecebe159d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.173546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72231c-e9b3-4104-aa37-b0faed1d1bba None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.174149 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72231c-e9b3-4104-aa37-b0faed1d1bba None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.174557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72231c-e9b3-4104-aa37-b0faed1d1bba None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.176387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 400/798] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:49.181616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72231c-e9b3-4104-aa37-b0faed1d1bba None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.183053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c1d49-8fa3-44fa-8b40-defecebe159d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:49.185085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c1d49-8fa3-44fa-8b40-defecebe159d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-a67c1d49-8fa3-44fa-8b40-defecebe159d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:49.202284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:49.203404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:17:49.204630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:49.207092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 399/799] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:17:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:49.208836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46556-9069-4414-9d17-5e9f112f4532 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.209282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46556-9069-4414-9d17-5e9f112f4532 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.210232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46556-9069-4414-9d17-5e9f112f4532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.232611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-6bc46556-9069-4414-9d17-5e9f112f4532 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:49.234635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6bc46556-9069-4414-9d17-5e9f112f4532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:49.300057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72231c-e9b3-4104-aa37-b0faed1d1bba None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:49.313407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 400/800] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:49.314562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc46556-9069-4414-9d17-5e9f112f4532 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.343898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211639a0-6c9b-49db-9693-c7fc1d11e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:49.344919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211639a0-6c9b-49db-9693-c7fc1d11e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.345522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211639a0-6c9b-49db-9693-c7fc1d11e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.346852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211639a0-6c9b-49db-9693-c7fc1d11e6e0 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.347818 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211639a0-6c9b-49db-9693-c7fc1d11e6e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:49.348332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211639a0-6c9b-49db-9693-c7fc1d11e6e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:49.350433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211639a0-6c9b-49db-9693-c7fc1d11e6e0 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:49.363144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 401/801] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:49.364316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211639a0-6c9b-49db-9693-c7fc1d11e6e0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.385684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:49.433370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:49.437604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc46556-9069-4414-9d17-5e9f112f4532 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:49.479671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211639a0-6c9b-49db-9693-c7fc1d11e6e0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:49.480785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65c364-1c78-4b02-a855-7520e7def365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:49.481395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65c364-1c78-4b02-a855-7520e7def365 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.481668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65c364-1c78-4b02-a855-7520e7def365 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.481938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f65c364-1c78-4b02-a855-7520e7def365 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.482829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65c364-1c78-4b02-a855-7520e7def365 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:49.483130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65c364-1c78-4b02-a855-7520e7def365 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:49.485357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f65c364-1c78-4b02-a855-7520e7def365 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:49.507267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 401/802] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 00:17:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:49.508576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65c364-1c78-4b02-a855-7520e7def365 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.519908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7ae0e-abe8-4b98-ab97-096d2e6b366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:49.520984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7ae0e-abe8-4b98-ab97-096d2e6b366c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.521515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7ae0e-abe8-4b98-ab97-096d2e6b366c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.522056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7ae0e-abe8-4b98-ab97-096d2e6b366c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.534170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7ae0e-abe8-4b98-ab97-096d2e6b366c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-28d7ae0e-abe8-4b98-ab97-096d2e6b366c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:49.540232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:49.541416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:49.542244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:49.545770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 402/803] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:49.547031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7ae0e-abe8-4b98-ab97-096d2e6b366c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.624859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f65c364-1c78-4b02-a855-7520e7def365 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:49.640481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:49.662403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154e4e0-79ea-4122-8916-99288622f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:49.663273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154e4e0-79ea-4122-8916-99288622f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.663547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154e4e0-79ea-4122-8916-99288622f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.663817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154e4e0-79ea-4122-8916-99288622f1d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.665233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7ae0e-abe8-4b98-ab97-096d2e6b366c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:49.670920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154e4e0-79ea-4122-8916-99288622f1d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:49.672382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154e4e0-79ea-4122-8916-99288622f1d4 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-7154e4e0-79ea-4122-8916-99288622f1d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:49.684325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:49.685256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:49.686124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:49.686124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:49.686124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:49.686124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:49.686124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:49.686124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:49.686124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:49.686124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:49.686124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 402/804] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:49.687212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154e4e0-79ea-4122-8916-99288622f1d4 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.706058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacc9c1-e129-4729-9e60-56a464d2db2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:49.706691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacc9c1-e129-4729-9e60-56a464d2db2a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.706968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacc9c1-e129-4729-9e60-56a464d2db2a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.707239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacc9c1-e129-4729-9e60-56a464d2db2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.709345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:49.719333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacc9c1-e129-4729-9e60-56a464d2db2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:49.721333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacc9c1-e129-4729-9e60-56a464d2db2a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-1eacc9c1-e129-4729-9e60-56a464d2db2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:49.736552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:49.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:49.738350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:49.738350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:49.738350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:49.738350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 403/805] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:49.739464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacc9c1-e129-4729-9e60-56a464d2db2a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.803721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154e4e0-79ea-4122-8916-99288622f1d4 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:49.839087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca152bb-2a52-46e5-8c80-fa1139930943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:49.839690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca152bb-2a52-46e5-8c80-fa1139930943 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.839960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca152bb-2a52-46e5-8c80-fa1139930943 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.840227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca152bb-2a52-46e5-8c80-fa1139930943 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.840906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca152bb-2a52-46e5-8c80-fa1139930943 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:49.841200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca152bb-2a52-46e5-8c80-fa1139930943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:49.842554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca152bb-2a52-46e5-8c80-fa1139930943 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:49.846401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacc9c1-e129-4729-9e60-56a464d2db2a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:49.856879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 403/806] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:49.856879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca152bb-2a52-46e5-8c80-fa1139930943 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.882127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:49.888886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffff59-e329-4385-87a8-e5ef97a62d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:49.889510 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffff59-e329-4385-87a8-e5ef97a62d52 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:49.889791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffff59-e329-4385-87a8-e5ef97a62d52 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:49.890061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffff59-e329-4385-87a8-e5ef97a62d52 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:49.890641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffff59-e329-4385-87a8-e5ef97a62d52 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:49.892000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffff59-e329-4385-87a8-e5ef97a62d52 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-d9ffff59-e329-4385-87a8-e5ef97a62d52 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:49.902552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:49.903562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:49.911138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 404/807] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 1 00:17:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:49.911950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffff59-e329-4385-87a8-e5ef97a62d52 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:49.941906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:49.971922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca152bb-2a52-46e5-8c80-fa1139930943 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.012878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3ab08-5471-4515-ba1f-89f633636d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.013661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3ab08-5471-4515-ba1f-89f633636d8b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.013946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3ab08-5471-4515-ba1f-89f633636d8b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.014218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3ab08-5471-4515-ba1f-89f633636d8b None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.014913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3ab08-5471-4515-ba1f-89f633636d8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:50.015220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3ab08-5471-4515-ba1f-89f633636d8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.016679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3ab08-5471-4515-ba1f-89f633636d8b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.028840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 404/808] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:50.029644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3ab08-5471-4515-ba1f-89f633636d8b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.032769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffff59-e329-4385-87a8-e5ef97a62d52 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.067863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f35fbb-4f95-4732-bc03-6a138d059087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.068467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f35fbb-4f95-4732-bc03-6a138d059087 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.068780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f35fbb-4f95-4732-bc03-6a138d059087 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.069049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f35fbb-4f95-4732-bc03-6a138d059087 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.069690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f35fbb-4f95-4732-bc03-6a138d059087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:50.069990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f35fbb-4f95-4732-bc03-6a138d059087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.071368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f35fbb-4f95-4732-bc03-6a138d059087 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.082605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 405/809] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:50.083393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f35fbb-4f95-4732-bc03-6a138d059087 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.134024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3ab08-5471-4515-ba1f-89f633636d8b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.138027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:50.166568 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481d6d3-351f-4986-8ef1-709156110bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.167183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481d6d3-351f-4986-8ef1-709156110bd5 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.167465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481d6d3-351f-4986-8ef1-709156110bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.167748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481d6d3-351f-4986-8ef1-709156110bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.174739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:50.179253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481d6d3-351f-4986-8ef1-709156110bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.180766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481d6d3-351f-4986-8ef1-709156110bd5 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.184253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f35fbb-4f95-4732-bc03-6a138d059087 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-5481d6d3-351f-4986-8ef1-709156110bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:50.195296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:50.196360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:50.197722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:50.197722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:50.197722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:50.197722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 405/810] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:50.197944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4b274-1ba7-4e26-8e94-37a4868db849 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.198186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4b274-1ba7-4e26-8e94-37a4868db849 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.198457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4b274-1ba7-4e26-8e94-37a4868db849 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.199688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 406/811] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:50.200358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4b274-1ba7-4e26-8e94-37a4868db849 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.223793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289ea20-6543-4a18-aeca-b9a3ba2fde3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.224415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289ea20-6543-4a18-aeca-b9a3ba2fde3e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.224712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289ea20-6543-4a18-aeca-b9a3ba2fde3e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.225231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289ea20-6543-4a18-aeca-b9a3ba2fde3e None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.225850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289ea20-6543-4a18-aeca-b9a3ba2fde3e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:50.226141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289ea20-6543-4a18-aeca-b9a3ba2fde3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.230048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289ea20-6543-4a18-aeca-b9a3ba2fde3e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.233858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 406/812] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 00:17:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:50.234644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289ea20-6543-4a18-aeca-b9a3ba2fde3e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.305109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4b274-1ba7-4e26-8e94-37a4868db849 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.337060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289ea20-6543-4a18-aeca-b9a3ba2fde3e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.339671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ff374-cf97-4a5a-a38d-21966d5d7770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.340332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ff374-cf97-4a5a-a38d-21966d5d7770 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.340647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ff374-cf97-4a5a-a38d-21966d5d7770 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.340920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ff374-cf97-4a5a-a38d-21966d5d7770 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/43d6c2a533e849cab240257cc1450c9f/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.342840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ff374-cf97-4a5a-a38d-21966d5d7770 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=43d6c2a533e849cab240257cc1450c9f, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.344291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ff374-cf97-4a5a-a38d-21966d5d7770 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.358115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 407/813] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:17:50 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/43d6c2a533e849cab240257cc1450c9f/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:50.359058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ff374-cf97-4a5a-a38d-21966d5d7770 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.360609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:50.382272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec51b01-6d47-4ef6-919e-604abced13c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.383504 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec51b01-6d47-4ef6-919e-604abced13c3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.383789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec51b01-6d47-4ef6-919e-604abced13c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.384115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec51b01-6d47-4ef6-919e-604abced13c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.391923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec51b01-6d47-4ef6-919e-604abced13c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.393721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec51b01-6d47-4ef6-919e-604abced13c3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-dec51b01-6d47-4ef6-919e-604abced13c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:50.406986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:50.408675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:50.409561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:50.409561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:50.409561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:50.409561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:50.409561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:50.409561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:50.409561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:50.409561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:50.410289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 407/814] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:50.411312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4a12b-2c30-477f-a086-cb627f08df59 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.411594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4a12b-2c30-477f-a086-cb627f08df59 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.411867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4a12b-2c30-477f-a086-cb627f08df59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.421094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:50.432270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-50d4a12b-2c30-477f-a086-cb627f08df59 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:50.433679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-50d4a12b-2c30-477f-a086-cb627f08df59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:50.482636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ff374-cf97-4a5a-a38d-21966d5d7770 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.515793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 408/815] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:50.517037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4a12b-2c30-477f-a086-cb627f08df59 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.521765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431ee6f-ab45-466b-8e15-d55c448de7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.522377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431ee6f-ab45-466b-8e15-d55c448de7a3 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.522652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431ee6f-ab45-466b-8e15-d55c448de7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.522919 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431ee6f-ab45-466b-8e15-d55c448de7a3 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.523559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431ee6f-ab45-466b-8e15-d55c448de7a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:50.523879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431ee6f-ab45-466b-8e15-d55c448de7a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.525291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431ee6f-ab45-466b-8e15-d55c448de7a3 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.538202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 408/816] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:50.538877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431ee6f-ab45-466b-8e15-d55c448de7a3 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.629466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4a12b-2c30-477f-a086-cb627f08df59 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.641455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:50.669659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431ee6f-ab45-466b-8e15-d55c448de7a3 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.673233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14878cd6-c9b8-43db-86a3-88b57c91e4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.674600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14878cd6-c9b8-43db-86a3-88b57c91e4f2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.674957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14878cd6-c9b8-43db-86a3-88b57c91e4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.675222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14878cd6-c9b8-43db-86a3-88b57c91e4f2 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.675848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14878cd6-c9b8-43db-86a3-88b57c91e4f2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:50.676145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14878cd6-c9b8-43db-86a3-88b57c91e4f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.677549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14878cd6-c9b8-43db-86a3-88b57c91e4f2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.684114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:50.687521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 409/817] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 1 00:17:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:50.688673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14878cd6-c9b8-43db-86a3-88b57c91e4f2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.713297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ecaea-d95f-483f-8f07-7177fad49372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.714389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ecaea-d95f-483f-8f07-7177fad49372 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.714863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ecaea-d95f-483f-8f07-7177fad49372 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.715328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ecaea-d95f-483f-8f07-7177fad49372 None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.716302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ecaea-d95f-483f-8f07-7177fad49372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:50.716857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ecaea-d95f-483f-8f07-7177fad49372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.718574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ecaea-d95f-483f-8f07-7177fad49372 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.732919 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 409/818] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:50.734051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ecaea-d95f-483f-8f07-7177fad49372 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.797013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14878cd6-c9b8-43db-86a3-88b57c91e4f2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.832900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f98d08-8022-4e05-82e8-b33872eae178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.833544 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f98d08-8022-4e05-82e8-b33872eae178 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.833808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f98d08-8022-4e05-82e8-b33872eae178 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.834119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f98d08-8022-4e05-82e8-b33872eae178 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.841719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f98d08-8022-4e05-82e8-b33872eae178 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.843155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f98d08-8022-4e05-82e8-b33872eae178 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.845071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ecaea-d95f-483f-8f07-7177fad49372 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-a9f98d08-8022-4e05-82e8-b33872eae178 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:50.854631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:50.856689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 410/819] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 00:17:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:50.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f98d08-8022-4e05-82e8-b33872eae178 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.887753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fbbde-d6b8-4372-9e49-365d146a0969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:50.888365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fbbde-d6b8-4372-9e49-365d146a0969 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:50.888673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fbbde-d6b8-4372-9e49-365d146a0969 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:50.888942 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fbbde-d6b8-4372-9e49-365d146a0969 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:50.889625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fbbde-d6b8-4372-9e49-365d146a0969 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:50.890965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fbbde-d6b8-4372-9e49-365d146a0969 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:50.894142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:50.914236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 410/820] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:17:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:50.925596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fbbde-d6b8-4372-9e49-365d146a0969 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:50.964317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:51.011498 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f98d08-8022-4e05-82e8-b33872eae178 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:51.070800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fbbde-d6b8-4372-9e49-365d146a0969 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:51.072402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b557f5-8f25-47a1-8ef9-775dcb605300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:51.073102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b557f5-8f25-47a1-8ef9-775dcb605300 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:51.073381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b557f5-8f25-47a1-8ef9-775dcb605300 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:51.073651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b557f5-8f25-47a1-8ef9-775dcb605300 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/6eb1b515c4a74ba5a9da1c94ec914a75/roles/87ec4f3044a84c9b8459c5458278b04c` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:51.091776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b557f5-8f25-47a1-8ef9-775dcb605300 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=6eb1b515c4a74ba5a9da1c94ec914a75, role_id=87ec4f3044a84c9b8459c5458278b04c)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:51.093313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b557f5-8f25-47a1-8ef9-775dcb605300 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:51.112452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 411/821] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:17:50 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/6eb1b515c4a74ba5a9da1c94ec914a75/roles/87ec4f3044a84c9b8459c5458278b04c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:17:51.113726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b557f5-8f25-47a1-8ef9-775dcb605300 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:51.124007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a788b55-de49-4aed-a6e5-ae2de32c6d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:51.124844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a788b55-de49-4aed-a6e5-ae2de32c6d1a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:51.125125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a788b55-de49-4aed-a6e5-ae2de32c6d1a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:51.125395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a788b55-de49-4aed-a6e5-ae2de32c6d1a None admin] PATH_INFO: `/v3/services/network` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:51.126533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a788b55-de49-4aed-a6e5-ae2de32c6d1a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:51.128027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a788b55-de49-4aed-a6e5-ae2de32c6d1a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-7a788b55-de49-4aed-a6e5-ae2de32c6d1a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:51.147773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:51.148891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:51.150257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 411/822] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:50 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:51.156756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a788b55-de49-4aed-a6e5-ae2de32c6d1a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:51.234794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:51.278268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:51.280030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b557f5-8f25-47a1-8ef9-775dcb605300 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:51.316780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a788b55-de49-4aed-a6e5-ae2de32c6d1a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:51.356545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec267d-7d98-4d40-a81d-3b3094508df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:51.357592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec267d-7d98-4d40-a81d-3b3094508df2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:51.358093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec267d-7d98-4d40-a81d-3b3094508df2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:51.358563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec267d-7d98-4d40-a81d-3b3094508df2 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:51.359504 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec267d-7d98-4d40-a81d-3b3094508df2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:51.360065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec267d-7d98-4d40-a81d-3b3094508df2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:51.362712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec267d-7d98-4d40-a81d-3b3094508df2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:51.370661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a456f6a-859c-4b4c-9c31-354927304594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:51.371278 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a456f6a-859c-4b4c-9c31-354927304594 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:51.371557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a456f6a-859c-4b4c-9c31-354927304594 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:51.371829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a456f6a-859c-4b4c-9c31-354927304594 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:51.372465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a456f6a-859c-4b4c-9c31-354927304594 None admin] RBAC: Adding query filter params (name=network) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:51.372874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a456f6a-859c-4b4c-9c31-354927304594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:51.374288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a456f6a-859c-4b4c-9c31-354927304594 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:51.380775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 412/823] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:17:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:51.386221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a456f6a-859c-4b4c-9c31-354927304594 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:51.387406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 412/824] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 00:17:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:51.392741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec267d-7d98-4d40-a81d-3b3094508df2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:51.506836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a456f6a-859c-4b4c-9c31-354927304594 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:51.518086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec267d-7d98-4d40-a81d-3b3094508df2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:51.558369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e691b25-6dd3-485b-a262-05ed65294bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:51.558982 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e691b25-6dd3-485b-a262-05ed65294bc2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:51.559263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e691b25-6dd3-485b-a262-05ed65294bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:51.559538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e691b25-6dd3-485b-a262-05ed65294bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:51.560092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbe4c4-e20f-41e1-ba0f-9821d54357d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:51.560892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbe4c4-e20f-41e1-ba0f-9821d54357d4 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:51.561327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbe4c4-e20f-41e1-ba0f-9821d54357d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:51.561749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbe4c4-e20f-41e1-ba0f-9821d54357d4 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:51.562809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbe4c4-e20f-41e1-ba0f-9821d54357d4 None admin] RBAC: Adding query filter params (type=network) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:51.564161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:51.565499 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbe4c4-e20f-41e1-ba0f-9821d54357d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:51.567045 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbe4c4-e20f-41e1-ba0f-9821d54357d4 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:51.573970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 413/825] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:17:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:51.581881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbe4c4-e20f-41e1-ba0f-9821d54357d4 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:51.614897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:51.664725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e691b25-6dd3-485b-a262-05ed65294bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:51.666817 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e691b25-6dd3-485b-a262-05ed65294bc2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:51.781114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 413/826] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:17:51.789806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e691b25-6dd3-485b-a262-05ed65294bc2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:51.823144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbe4c4-e20f-41e1-ba0f-9821d54357d4 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:51.840668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:51.869680 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d42e0-a783-4d45-b0cc-18cf58c05c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:51.873185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d42e0-a783-4d45-b0cc-18cf58c05c5d None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:51.873672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d42e0-a783-4d45-b0cc-18cf58c05c5d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:51.874182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d42e0-a783-4d45-b0cc-18cf58c05c5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:51.875053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d42e0-a783-4d45-b0cc-18cf58c05c5d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:51.876677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d42e0-a783-4d45-b0cc-18cf58c05c5d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:51.925922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 414/827] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:17:51 2022] POST /identity/v3/endpoints => generated 337 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:51.930860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e691b25-6dd3-485b-a262-05ed65294bc2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:51.981166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7813-4965-47fe-a94d-7a92771930cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:51.982194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7813-4965-47fe-a94d-7a92771930cf None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:51.982675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7813-4965-47fe-a94d-7a92771930cf None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:51.983165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7813-4965-47fe-a94d-7a92771930cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:51.995837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d7813-4965-47fe-a94d-7a92771930cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:51.997709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d7813-4965-47fe-a94d-7a92771930cf None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f95d7813-4965-47fe-a94d-7a92771930cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:52.015258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:52.016318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:52.017241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:52.017241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:52.017241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:52.019969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 414/828] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:52.025389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d42e0-a783-4d45-b0cc-18cf58c05c5d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:52.142636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:52.180869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d42e0-a783-4d45-b0cc-18cf58c05c5d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:52.217793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916c2b8-667f-4812-ad30-c070a6d47d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:52.218462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916c2b8-667f-4812-ad30-c070a6d47d26 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:52.218744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916c2b8-667f-4812-ad30-c070a6d47d26 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:52.219015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916c2b8-667f-4812-ad30-c070a6d47d26 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:52.219717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916c2b8-667f-4812-ad30-c070a6d47d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:52.220022 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916c2b8-667f-4812-ad30-c070a6d47d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:52.221550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916c2b8-667f-4812-ad30-c070a6d47d26 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:52.236591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 415/829] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:52.240643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7813-4965-47fe-a94d-7a92771930cf None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:52.339053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:52.358020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7813-4965-47fe-a94d-7a92771930cf None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:52.395091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0bbfaf-4671-4cb4-add1-9ef4833f71a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:52.395651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bbfaf-4671-4cb4-add1-9ef4833f71a6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:52.395929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bbfaf-4671-4cb4-add1-9ef4833f71a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:52.396201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0bbfaf-4671-4cb4-add1-9ef4833f71a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:52.403481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bbfaf-4671-4cb4-add1-9ef4833f71a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:52.405007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0bbfaf-4671-4cb4-add1-9ef4833f71a6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-bd0bbfaf-4671-4cb4-add1-9ef4833f71a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:52.418701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:52.419602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:52.420486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:52.420486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:52.420486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:52.420486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:52.420486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:52.420486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:52.420486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:52.420486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:52.420486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 415/830] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:52.426709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916c2b8-667f-4812-ad30-c070a6d47d26 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:52.543904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916c2b8-667f-4812-ad30-c070a6d47d26 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:52.559381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773cc8a-0981-4c4d-892b-bdf9df461e55 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:52.559656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773cc8a-0981-4c4d-892b-bdf9df461e55 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:52.559937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773cc8a-0981-4c4d-892b-bdf9df461e55 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:52.561549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 416/831] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:17:52.567395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c3836-8b84-4f42-8617-b7d9296cfd14 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:52.567682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c3836-8b84-4f42-8617-b7d9296cfd14 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:52.567952 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c3836-8b84-4f42-8617-b7d9296cfd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:52.575744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:52.583027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215c6f2-fd66-4223-a089-4a1396aa775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:52.583999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215c6f2-fd66-4223-a089-4a1396aa775e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:52.584462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215c6f2-fd66-4223-a089-4a1396aa775e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:52.584946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215c6f2-fd66-4223-a089-4a1396aa775e None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:52.585856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215c6f2-fd66-4223-a089-4a1396aa775e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:52.586338 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215c6f2-fd66-4223-a089-4a1396aa775e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:52.587799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f32c3836-8b84-4f42-8617-b7d9296cfd14 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:52.588399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c215c6f2-fd66-4223-a089-4a1396aa775e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:52.589085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f32c3836-8b84-4f42-8617-b7d9296cfd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:52.603287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 416/832] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:52.612655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215c6f2-fd66-4223-a089-4a1396aa775e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:52.661679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 417/833] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:52.727699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215c6f2-fd66-4223-a089-4a1396aa775e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:52.763446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6986c-30c7-4c16-8ad6-13fd84e65852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:52.764430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6986c-30c7-4c16-8ad6-13fd84e65852 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:52.764936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6986c-30c7-4c16-8ad6-13fd84e65852 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:52.765426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6986c-30c7-4c16-8ad6-13fd84e65852 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:52.766355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6986c-30c7-4c16-8ad6-13fd84e65852 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=ed93469862874c75951d95e28929c904, user.id=fb204040ab2f4168b35c3e78bacb849f) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:52.766886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6986c-30c7-4c16-8ad6-13fd84e65852 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:52.768651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6986c-30c7-4c16-8ad6-13fd84e65852 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:52.775303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 417/834] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:52 2022] GET /identity/v3/role_assignments?user.id=fb204040ab2f4168b35c3e78bacb849f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:52.955898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c3836-8b84-4f42-8617-b7d9296cfd14 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:53.045948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:53.073937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c3836-8b84-4f42-8617-b7d9296cfd14 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:53.113832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b2a10-f5ea-4d64-863b-71bfc314b0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:53.114468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2a10-f5ea-4d64-863b-71bfc314b0fc None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:53.114741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2a10-f5ea-4d64-863b-71bfc314b0fc None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:53.115008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2a10-f5ea-4d64-863b-71bfc314b0fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:53.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b2a10-f5ea-4d64-863b-71bfc314b0fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f46b2a10-f5ea-4d64-863b-71bfc314b0fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:53.131711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:53.132666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:53.133525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 418/835] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:53.141739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6986c-30c7-4c16-8ad6-13fd84e65852 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:53.207412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:53.270624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6986c-30c7-4c16-8ad6-13fd84e65852 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:53.310205 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38f7ae-ace3-428d-82e1-0a86b93f7b0f None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:53.310718 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38f7ae-ace3-428d-82e1-0a86b93f7b0f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:53.310718 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38f7ae-ace3-428d-82e1-0a86b93f7b0f None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:53.311175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb346a92-e347-4ca6-8250-a51a2e5b1fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:53.312086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb346a92-e347-4ca6-8250-a51a2e5b1fe8 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:53.312404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 419/836] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:53.312811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb346a92-e347-4ca6-8250-a51a2e5b1fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:53.313104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb346a92-e347-4ca6-8250-a51a2e5b1fe8 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:53.313875 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb346a92-e347-4ca6-8250-a51a2e5b1fe8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:53.314126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb346a92-e347-4ca6-8250-a51a2e5b1fe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:53.315620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb346a92-e347-4ca6-8250-a51a2e5b1fe8 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:53.323949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2c112-7233-4af5-85e5-4cd3e384fc93 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:53.324261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2c112-7233-4af5-85e5-4cd3e384fc93 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:53.324759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2c112-7233-4af5-85e5-4cd3e384fc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:53.327740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 418/837] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:17:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:53.333712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb346a92-e347-4ca6-8250-a51a2e5b1fe8 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:53.351301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-caf2c112-7233-4af5-85e5-4cd3e384fc93 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:53.352911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-caf2c112-7233-4af5-85e5-4cd3e384fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:53.445781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb346a92-e347-4ca6-8250-a51a2e5b1fe8 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:53.462509 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 420/838] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:17:53.464440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3cd7a-18b4-4021-bcc6-90120f72337e None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:53.465060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3cd7a-18b4-4021-bcc6-90120f72337e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:53.465541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3cd7a-18b4-4021-bcc6-90120f72337e None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:53.467248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:53.467697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 421/839] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:53.479253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aa20f-7b7b-4650-9a9c-26da31eca1f3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:53.479771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aa20f-7b7b-4650-9a9c-26da31eca1f3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:53.480248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aa20f-7b7b-4650-9a9c-26da31eca1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:53.490029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52ee4b-3f2a-46ce-9bae-08999dd2cb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:53.490937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52ee4b-3f2a-46ce-9bae-08999dd2cb7a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:53.491412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52ee4b-3f2a-46ce-9bae-08999dd2cb7a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:53.491849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52ee4b-3f2a-46ce-9bae-08999dd2cb7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:53.501417 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-2a2aa20f-7b7b-4650-9a9c-26da31eca1f3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:53.503254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2a2aa20f-7b7b-4650-9a9c-26da31eca1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:53.504410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52ee4b-3f2a-46ce-9bae-08999dd2cb7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:53.506415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52ee4b-3f2a-46ce-9bae-08999dd2cb7a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-ea52ee4b-3f2a-46ce-9bae-08999dd2cb7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:53.523424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:53.524528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:53.525413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:53.525413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:53.525413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:53.527602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 419/840] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:53.536558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52ee4b-3f2a-46ce-9bae-08999dd2cb7a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:53.573974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:53.592891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 422/841] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:53.657741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52ee4b-3f2a-46ce-9bae-08999dd2cb7a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:53.694947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e60f9-e30b-4292-bf91-317ebd6d746a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:53.695548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e60f9-e30b-4292-bf91-317ebd6d746a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:53.695822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e60f9-e30b-4292-bf91-317ebd6d746a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:53.696090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e60f9-e30b-4292-bf91-317ebd6d746a None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:53.696758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e60f9-e30b-4292-bf91-317ebd6d746a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:53.697059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e60f9-e30b-4292-bf91-317ebd6d746a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:53.698543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e60f9-e30b-4292-bf91-317ebd6d746a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:53.711910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 420/842] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:53.717580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2aa20f-7b7b-4650-9a9c-26da31eca1f3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:53.828361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2aa20f-7b7b-4650-9a9c-26da31eca1f3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:53.828914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e60f9-e30b-4292-bf91-317ebd6d746a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:53.868092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:53.871566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8626e-418b-4ec6-8bee-1fe02e7a5b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:53.872591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8626e-418b-4ec6-8bee-1fe02e7a5b79 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:53.873138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8626e-418b-4ec6-8bee-1fe02e7a5b79 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:53.873464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8626e-418b-4ec6-8bee-1fe02e7a5b79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:53.884202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8626e-418b-4ec6-8bee-1fe02e7a5b79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:53.885757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8626e-418b-4ec6-8bee-1fe02e7a5b79 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-89f8626e-418b-4ec6-8bee-1fe02e7a5b79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:53.899230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:53.900746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 00:17:53.901841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:53.903876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 423/843] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:17:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:53.904091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8626e-418b-4ec6-8bee-1fe02e7a5b79 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:53.958838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e60f9-e30b-4292-bf91-317ebd6d746a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:53.986490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:54.009261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333393f9-6361-4a9e-9a09-981632ae9de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.010425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333393f9-6361-4a9e-9a09-981632ae9de6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.010707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333393f9-6361-4a9e-9a09-981632ae9de6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.010979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333393f9-6361-4a9e-9a09-981632ae9de6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.011573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333393f9-6361-4a9e-9a09-981632ae9de6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:54.013339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333393f9-6361-4a9e-9a09-981632ae9de6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-333393f9-6361-4a9e-9a09-981632ae9de6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:54.023933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:54.025072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:54.025898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 421/844] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:54.026697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333393f9-6361-4a9e-9a09-981632ae9de6 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:54.044959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8626e-418b-4ec6-8bee-1fe02e7a5b79 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.081867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923351fb-4d22-4300-889a-e791b2a4e22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.082484 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923351fb-4d22-4300-889a-e791b2a4e22c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.082758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923351fb-4d22-4300-889a-e791b2a4e22c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.083025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923351fb-4d22-4300-889a-e791b2a4e22c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.093785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923351fb-4d22-4300-889a-e791b2a4e22c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-923351fb-4d22-4300-889a-e791b2a4e22c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:54.099098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:54.100262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:54.101136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 424/845] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:17:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:54.101849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923351fb-4d22-4300-889a-e791b2a4e22c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:54.137864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:54.149668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333393f9-6361-4a9e-9a09-981632ae9de6 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.192618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587c3f0-7eb7-4556-a100-68c557cd0c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.193381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587c3f0-7eb7-4556-a100-68c557cd0c7d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.193664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587c3f0-7eb7-4556-a100-68c557cd0c7d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.194249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587c3f0-7eb7-4556-a100-68c557cd0c7d None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.195248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587c3f0-7eb7-4556-a100-68c557cd0c7d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:54.195554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587c3f0-7eb7-4556-a100-68c557cd0c7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:54.197494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587c3f0-7eb7-4556-a100-68c557cd0c7d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:54.202466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923351fb-4d22-4300-889a-e791b2a4e22c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.220551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 422/846] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:54.221045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:54.221149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587c3f0-7eb7-4556-a100-68c557cd0c7d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:54.247007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12814f-6fb7-4e88-8e9b-e543f76e242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.247624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12814f-6fb7-4e88-8e9b-e543f76e242d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.247828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12814f-6fb7-4e88-8e9b-e543f76e242d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.248079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12814f-6fb7-4e88-8e9b-e543f76e242d None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.251397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12814f-6fb7-4e88-8e9b-e543f76e242d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:54.251397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12814f-6fb7-4e88-8e9b-e543f76e242d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:54.251397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12814f-6fb7-4e88-8e9b-e543f76e242d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:54.256202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 425/847] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:54.262085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12814f-6fb7-4e88-8e9b-e543f76e242d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:54.339791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587c3f0-7eb7-4556-a100-68c557cd0c7d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.372440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12814f-6fb7-4e88-8e9b-e543f76e242d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.374298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c361f-20e4-428c-9357-0a61a3d93659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.374898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c361f-20e4-428c-9357-0a61a3d93659 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.375173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c361f-20e4-428c-9357-0a61a3d93659 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.375442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c361f-20e4-428c-9357-0a61a3d93659 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.376099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c361f-20e4-428c-9357-0a61a3d93659 None admin] RBAC: Adding query filter params (name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:54.376391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c361f-20e4-428c-9357-0a61a3d93659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:54.378397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c361f-20e4-428c-9357-0a61a3d93659 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:54.386884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 423/848] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:54.387588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c361f-20e4-428c-9357-0a61a3d93659 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:54.404290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:54.415826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40e21a-1cee-4b28-9e07-9d08a2f24702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.416436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40e21a-1cee-4b28-9e07-9d08a2f24702 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.416759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40e21a-1cee-4b28-9e07-9d08a2f24702 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.417028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40e21a-1cee-4b28-9e07-9d08a2f24702 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.467078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:54.493359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40e21a-1cee-4b28-9e07-9d08a2f24702 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:54.494847 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40e21a-1cee-4b28-9e07-9d08a2f24702 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:54.496924 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c361f-20e4-428c-9357-0a61a3d93659 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.533447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb560cf-2d13-4da9-be91-aae57172bb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.533966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb560cf-2d13-4da9-be91-aae57172bb59 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.534238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb560cf-2d13-4da9-be91-aae57172bb59 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.534503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb560cf-2d13-4da9-be91-aae57172bb59 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.535129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb560cf-2d13-4da9-be91-aae57172bb59 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:54.535426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb560cf-2d13-4da9-be91-aae57172bb59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:54.536828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb560cf-2d13-4da9-be91-aae57172bb59 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:54.548939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 424/849] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:54.548939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb560cf-2d13-4da9-be91-aae57172bb59 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:54.630227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 426/850] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:17:54.635727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40e21a-1cee-4b28-9e07-9d08a2f24702 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:54.659430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:54.688232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb560cf-2d13-4da9-be91-aae57172bb59 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.732719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2ab14-4d35-4ac2-b1f5-ba41d57ff37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.732719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2ab14-4d35-4ac2-b1f5-ba41d57ff37d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.732719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2ab14-4d35-4ac2-b1f5-ba41d57ff37d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.732719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2ab14-4d35-4ac2-b1f5-ba41d57ff37d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.733176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:54.748067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2ab14-4d35-4ac2-b1f5-ba41d57ff37d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:54.750768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2ab14-4d35-4ac2-b1f5-ba41d57ff37d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:54.759013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40e21a-1cee-4b28-9e07-9d08a2f24702 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-d7e2ab14-4d35-4ac2-b1f5-ba41d57ff37d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:54.771429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:54.772622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:54.773736 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:54.773736 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:54.773736 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:54.776205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 425/851] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:54.782177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2ab14-4d35-4ac2-b1f5-ba41d57ff37d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:54.807481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ff2d1-7d65-4d9a-acda-8e9dd9e33d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.808525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ff2d1-7d65-4d9a-acda-8e9dd9e33d24 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.808829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ff2d1-7d65-4d9a-acda-8e9dd9e33d24 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.809101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ff2d1-7d65-4d9a-acda-8e9dd9e33d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.823354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ff2d1-7d65-4d9a-acda-8e9dd9e33d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:54.825633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ff2d1-7d65-4d9a-acda-8e9dd9e33d24 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-113ff2d1-7d65-4d9a-acda-8e9dd9e33d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:54.845839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:54.846933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:54.847819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:54.847819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:54.847819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:54.847819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 427/852] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:54.853153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ff2d1-7d65-4d9a-acda-8e9dd9e33d24 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:54.910029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2ab14-4d35-4ac2-b1f5-ba41d57ff37d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.937891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:54.947129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057c751-99e1-4655-b10e-3c7f1d237bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:54.947642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057c751-99e1-4655-b10e-3c7f1d237bc2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.947923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057c751-99e1-4655-b10e-3c7f1d237bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.948200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057c751-99e1-4655-b10e-3c7f1d237bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.948869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057c751-99e1-4655-b10e-3c7f1d237bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:54.949175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057c751-99e1-4655-b10e-3c7f1d237bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:54.950558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057c751-99e1-4655-b10e-3c7f1d237bc2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:54.967432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 426/853] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:54.968619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a0d9f-cd9a-4d08-b560-457360e4c489 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:54.968724 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ff2d1-7d65-4d9a-acda-8e9dd9e33d24 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:54.968908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a0d9f-cd9a-4d08-b560-457360e4c489 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:54.969179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a0d9f-cd9a-4d08-b560-457360e4c489 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:54.971020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 427/854] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:54.971877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a0d9f-cd9a-4d08-b560-457360e4c489 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.005436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:55.011885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0075a4-973d-4d5f-8c0e-739ddb3cb509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:55.012525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0075a4-973d-4d5f-8c0e-739ddb3cb509 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.012843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0075a4-973d-4d5f-8c0e-739ddb3cb509 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.013068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0075a4-973d-4d5f-8c0e-739ddb3cb509 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.013717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0075a4-973d-4d5f-8c0e-739ddb3cb509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:55.014016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0075a4-973d-4d5f-8c0e-739ddb3cb509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:55.015399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0075a4-973d-4d5f-8c0e-739ddb3cb509 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:55.033961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 428/855] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:55.033961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e707f-cab0-41ee-aab9-0ca697c13fc6 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.033961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e707f-cab0-41ee-aab9-0ca697c13fc6 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.033961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e707f-cab0-41ee-aab9-0ca697c13fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.050529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-926e707f-cab0-41ee-aab9-0ca697c13fc6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:55.051727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-926e707f-cab0-41ee-aab9-0ca697c13fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:55.091220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a0d9f-cd9a-4d08-b560-457360e4c489 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:55.121793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 429/856] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:55.122472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e707f-cab0-41ee-aab9-0ca697c13fc6 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.135529 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f9845-4711-4eeb-bd39-b3fa7a59e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:55.137165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f9845-4711-4eeb-bd39-b3fa7a59e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.137704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f9845-4711-4eeb-bd39-b3fa7a59e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.138173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f9845-4711-4eeb-bd39-b3fa7a59e9a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.148155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f9845-4711-4eeb-bd39-b3fa7a59e9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:55.150850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f9845-4711-4eeb-bd39-b3fa7a59e9a9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-253f9845-4711-4eeb-bd39-b3fa7a59e9a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:17:55.168318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:55.169349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:17:55.170287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:55.172981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 428/857] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:17:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:55.179658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f9845-4711-4eeb-bd39-b3fa7a59e9a9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.233805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:55.242985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e707f-cab0-41ee-aab9-0ca697c13fc6 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:55.268610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:55.284134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edc9d0-04dc-4f67-b9fd-616b9ffbb530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:55.284914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edc9d0-04dc-4f67-b9fd-616b9ffbb530 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.285267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edc9d0-04dc-4f67-b9fd-616b9ffbb530 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.285614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edc9d0-04dc-4f67-b9fd-616b9ffbb530 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.292763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edc9d0-04dc-4f67-b9fd-616b9ffbb530 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:55.294338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edc9d0-04dc-4f67-b9fd-616b9ffbb530 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:55.296362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f9845-4711-4eeb-bd39-b3fa7a59e9a9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-d6edc9d0-04dc-4f67-b9fd-616b9ffbb530 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:55.307927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:55.308877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:55.309756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:55.309756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:55.309756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:55.309756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:55.309756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:55.309756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:55.309756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:55.309756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:55.309756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 430/858] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:55.315275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edc9d0-04dc-4f67-b9fd-616b9ffbb530 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.342199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f3628-bbcc-480e-af37-59b994a8ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:55.342952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f3628-bbcc-480e-af37-59b994a8ba4c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.343236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f3628-bbcc-480e-af37-59b994a8ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.343514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f3628-bbcc-480e-af37-59b994a8ba4c None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.344596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f3628-bbcc-480e-af37-59b994a8ba4c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:55.344908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f3628-bbcc-480e-af37-59b994a8ba4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:55.348066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f3628-bbcc-480e-af37-59b994a8ba4c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:55.377173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 429/859] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 00:17:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:55.381662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f3628-bbcc-480e-af37-59b994a8ba4c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.430985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edc9d0-04dc-4f67-b9fd-616b9ffbb530 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:55.465855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d231f-4d52-4505-8f2c-8954c9fee919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:55.466454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d231f-4d52-4505-8f2c-8954c9fee919 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.466731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d231f-4d52-4505-8f2c-8954c9fee919 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.467005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d231f-4d52-4505-8f2c-8954c9fee919 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.467700 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d231f-4d52-4505-8f2c-8954c9fee919 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:55.468025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d231f-4d52-4505-8f2c-8954c9fee919 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:55.469492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d231f-4d52-4505-8f2c-8954c9fee919 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:55.487603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 431/860] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:55.488566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d231f-4d52-4505-8f2c-8954c9fee919 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.495320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f3628-bbcc-480e-af37-59b994a8ba4c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:55.505098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:55.535372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588380d-dafd-42ec-8e34-1b0ce801ec33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:55.535976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588380d-dafd-42ec-8e34-1b0ce801ec33 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.536438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:55.536640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588380d-dafd-42ec-8e34-1b0ce801ec33 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.536911 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588380d-dafd-42ec-8e34-1b0ce801ec33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.603797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d231f-4d52-4505-8f2c-8954c9fee919 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:55.608983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588380d-dafd-42ec-8e34-1b0ce801ec33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:55.610438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588380d-dafd-42ec-8e34-1b0ce801ec33 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:55.639871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a446cb-b0ff-4b35-b255-d14616b67da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:55.640534 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a446cb-b0ff-4b35-b255-d14616b67da1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.640816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a446cb-b0ff-4b35-b255-d14616b67da1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.641092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a446cb-b0ff-4b35-b255-d14616b67da1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.641692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a446cb-b0ff-4b35-b255-d14616b67da1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:55.643423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a446cb-b0ff-4b35-b255-d14616b67da1 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-04a446cb-b0ff-4b35-b255-d14616b67da1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:55.659759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:55.660801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:55.661690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 432/861] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 1 00:17:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:55.662456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a446cb-b0ff-4b35-b255-d14616b67da1 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.726948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 430/862] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:17:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 00:17:55.728091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588380d-dafd-42ec-8e34-1b0ce801ec33 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.737752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:55.771862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a446cb-b0ff-4b35-b255-d14616b67da1 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:55.775563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:55.809768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86e716-f3e5-42f7-8f1b-c53765a9de44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:55.810766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86e716-f3e5-42f7-8f1b-c53765a9de44 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.811040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86e716-f3e5-42f7-8f1b-c53765a9de44 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.811306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86e716-f3e5-42f7-8f1b-c53765a9de44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.812384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86e716-f3e5-42f7-8f1b-c53765a9de44 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=ed93469862874c75951d95e28929c904, user.id=43d6c2a533e849cab240257cc1450c9f) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:55.812720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86e716-f3e5-42f7-8f1b-c53765a9de44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:55.815317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86e716-f3e5-42f7-8f1b-c53765a9de44 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:55.821667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 433/863] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:55 2022] GET /identity/v3/role_assignments?user.id=43d6c2a533e849cab240257cc1450c9f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:55.829857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86e716-f3e5-42f7-8f1b-c53765a9de44 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.836216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588380d-dafd-42ec-8e34-1b0ce801ec33 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:55.876946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc551d12-a23f-4691-b52d-105007866040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:55.877657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc551d12-a23f-4691-b52d-105007866040 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:55.878036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc551d12-a23f-4691-b52d-105007866040 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:55.878542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc551d12-a23f-4691-b52d-105007866040 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:55.879174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc551d12-a23f-4691-b52d-105007866040 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:55.879574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc551d12-a23f-4691-b52d-105007866040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:55.881162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc551d12-a23f-4691-b52d-105007866040 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:55.891256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 431/864] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:17:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:55.893867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc551d12-a23f-4691-b52d-105007866040 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:55.981778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86e716-f3e5-42f7-8f1b-c53765a9de44 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:55.985648 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:56.022417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc551d12-a23f-4691-b52d-105007866040 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:56.031815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91206684-f678-4a42-823e-cb39c961bf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:56.032468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91206684-f678-4a42-823e-cb39c961bf6c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:56.033254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91206684-f678-4a42-823e-cb39c961bf6c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:56.033553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91206684-f678-4a42-823e-cb39c961bf6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:56.053148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91206684-f678-4a42-823e-cb39c961bf6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:56.055864 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91206684-f678-4a42-823e-cb39c961bf6c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:56.062518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ccbec-9c46-4a0f-b13b-0f25d3135931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:56.063125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ccbec-9c46-4a0f-b13b-0f25d3135931 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:56.063464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ccbec-9c46-4a0f-b13b-0f25d3135931 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:56.063795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ccbec-9c46-4a0f-b13b-0f25d3135931 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:56.064542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ccbec-9c46-4a0f-b13b-0f25d3135931 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:56.064895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ccbec-9c46-4a0f-b13b-0f25d3135931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:56.066405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ccbec-9c46-4a0f-b13b-0f25d3135931 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:56.071945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 432/865] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:17:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-91206684-f678-4a42-823e-cb39c961bf6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:56.075688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:56.076897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:56.077792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:56.077792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:56.077792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:56.077792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 434/866] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:56.088569 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ccbec-9c46-4a0f-b13b-0f25d3135931 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:56.201479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ccbec-9c46-4a0f-b13b-0f25d3135931 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:56.240325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218d25a-7119-46de-a4d0-38b5c4cbaa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:56.240975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218d25a-7119-46de-a4d0-38b5c4cbaa25 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:56.241240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218d25a-7119-46de-a4d0-38b5c4cbaa25 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:56.241509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218d25a-7119-46de-a4d0-38b5c4cbaa25 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:56.242152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218d25a-7119-46de-a4d0-38b5c4cbaa25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:56.242451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218d25a-7119-46de-a4d0-38b5c4cbaa25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:56.243807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218d25a-7119-46de-a4d0-38b5c4cbaa25 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:56.246523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:56.258111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 433/867] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:56.263580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91206684-f678-4a42-823e-cb39c961bf6c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:56.287426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:56.371254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91206684-f678-4a42-823e-cb39c961bf6c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:56.406296 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1c81b-79c6-4a3d-959d-db60b3612666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:56.406917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1c81b-79c6-4a3d-959d-db60b3612666 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:56.407187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1c81b-79c6-4a3d-959d-db60b3612666 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:56.407452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1c81b-79c6-4a3d-959d-db60b3612666 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:56.414895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1c81b-79c6-4a3d-959d-db60b3612666 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:56.416337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1c81b-79c6-4a3d-959d-db60b3612666 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-6df1c81b-79c6-4a3d-959d-db60b3612666 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:17:56.427874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:56.428830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:56.429707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:17:56.429707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:17:56.429707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:17:56.429707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:17:56.429707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:17:56.429707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:17:56.429707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:17:56.429707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:56.429707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 435/868] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:17:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:56.434971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218d25a-7119-46de-a4d0-38b5c4cbaa25 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:56.497353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca521d-a4ff-41e9-87eb-74cb1bfa3254 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:56.497843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca521d-a4ff-41e9-87eb-74cb1bfa3254 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:56.497843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca521d-a4ff-41e9-87eb-74cb1bfa3254 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:56.499339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 436/869] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:56.505681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e11c06-f516-43dc-82b6-9c630a8d5f73 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:56.505950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e11c06-f516-43dc-82b6-9c630a8d5f73 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:56.506213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e11c06-f516-43dc-82b6-9c630a8d5f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:56.528642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a6e11c06-f516-43dc-82b6-9c630a8d5f73 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:56.530276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a6e11c06-f516-43dc-82b6-9c630a8d5f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:56.542692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218d25a-7119-46de-a4d0-38b5c4cbaa25 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:56.585287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88b269-d5c2-40c0-8b80-1053bc606f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:56.586266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88b269-d5c2-40c0-8b80-1053bc606f2f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:56.586813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88b269-d5c2-40c0-8b80-1053bc606f2f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:56.587093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88b269-d5c2-40c0-8b80-1053bc606f2f None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:56.587917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88b269-d5c2-40c0-8b80-1053bc606f2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:56.588243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88b269-d5c2-40c0-8b80-1053bc606f2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:56.589979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88b269-d5c2-40c0-8b80-1053bc606f2f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:56.608825 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 434/870] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:17:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:56.612537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88b269-d5c2-40c0-8b80-1053bc606f2f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:56.613831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 437/871] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:56.675185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:56.721244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88b269-d5c2-40c0-8b80-1053bc606f2f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:56.756748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d95495-6dcb-4f10-bf70-03191f164cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:56.757269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d95495-6dcb-4f10-bf70-03191f164cf4 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:56.757539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d95495-6dcb-4f10-bf70-03191f164cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:56.757809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d95495-6dcb-4f10-bf70-03191f164cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:56.758588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d95495-6dcb-4f10-bf70-03191f164cf4 None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=6eb1b515c4a74ba5a9da1c94ec914a75) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:56.758894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d95495-6dcb-4f10-bf70-03191f164cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:56.760323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d95495-6dcb-4f10-bf70-03191f164cf4 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:56.766927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 435/872] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:17:56 2022] GET /identity/v3/role_assignments?user.id=6eb1b515c4a74ba5a9da1c94ec914a75&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:56.912730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e11c06-f516-43dc-82b6-9c630a8d5f73 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:56.925394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:57.024799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e11c06-f516-43dc-82b6-9c630a8d5f73 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:57.059968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55659e-b064-40e6-96c8-eba70d09d0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:57.060611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55659e-b064-40e6-96c8-eba70d09d0a3 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:57.060886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55659e-b064-40e6-96c8-eba70d09d0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:57.061158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55659e-b064-40e6-96c8-eba70d09d0a3 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:57.061802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55659e-b064-40e6-96c8-eba70d09d0a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:57.063163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55659e-b064-40e6-96c8-eba70d09d0a3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:57.081843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 438/873] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:17:56 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:17:57.842774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9379c-edd6-4662-978e-4d2dcf1faa47 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:57.843290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9379c-edd6-4662-978e-4d2dcf1faa47 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:57.843290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9379c-edd6-4662-978e-4d2dcf1faa47 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:57.844943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 436/874] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:57.850762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d8651-fa63-474d-8e1b-115100fb8fc9 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:57.851183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d8651-fa63-474d-8e1b-115100fb8fc9 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:57.851283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d8651-fa63-474d-8e1b-115100fb8fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:57.869600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a62d8651-fa63-474d-8e1b-115100fb8fc9 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:57.870791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a62d8651-fa63-474d-8e1b-115100fb8fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:57.936627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:57.948546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 439/875] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:58.061681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98194c2d-2e1d-408f-ae27-178291d58396 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:58.062008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98194c2d-2e1d-408f-ae27-178291d58396 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:58.062238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98194c2d-2e1d-408f-ae27-178291d58396 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:58.064350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 437/876] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:58.069496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc53e5a-48f2-404d-835a-0688ecc91738 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:58.069782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc53e5a-48f2-404d-835a-0688ecc91738 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:58.070078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc53e5a-48f2-404d-835a-0688ecc91738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:58.089364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bbc53e5a-48f2-404d-835a-0688ecc91738 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:58.090548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bbc53e5a-48f2-404d-835a-0688ecc91738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:58.159943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 440/877] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:58 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:58.225108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98194c2d-2e1d-408f-ae27-178291d58396 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:58.330661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98194c2d-2e1d-408f-ae27-178291d58396 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:58.348079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:58.366500 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec721ced-67f2-49a9-916b-f6b7b5e99b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:58.367122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec721ced-67f2-49a9-916b-f6b7b5e99b70 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:58.367399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec721ced-67f2-49a9-916b-f6b7b5e99b70 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:58.367667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec721ced-67f2-49a9-916b-f6b7b5e99b70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:58.368244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec721ced-67f2-49a9-916b-f6b7b5e99b70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:58.369816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec721ced-67f2-49a9-916b-f6b7b5e99b70 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-ec721ced-67f2-49a9-916b-f6b7b5e99b70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:58.378863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:58.379816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:58.380674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 438/878] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:17:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:17:58.386173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc53e5a-48f2-404d-835a-0688ecc91738 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:58.444618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec721ced-67f2-49a9-916b-f6b7b5e99b70 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:58.495635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc53e5a-48f2-404d-835a-0688ecc91738 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:58.530662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:58.543374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6919b-1c81-4788-988e-3b541b57af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:58.543979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6919b-1c81-4788-988e-3b541b57af5b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:58.544248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6919b-1c81-4788-988e-3b541b57af5b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:58.544535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6919b-1c81-4788-988e-3b541b57af5b None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:58.545170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6919b-1c81-4788-988e-3b541b57af5b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:58.545894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6919b-1c81-4788-988e-3b541b57af5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:58.547555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6919b-1c81-4788-988e-3b541b57af5b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:58.556628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 441/879] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:17:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:58.560629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6919b-1c81-4788-988e-3b541b57af5b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:58.563444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec721ced-67f2-49a9-916b-f6b7b5e99b70 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:58.602624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa15a1b-2b1a-4a3c-88b4-7f1416c327ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:58.603228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa15a1b-2b1a-4a3c-88b4-7f1416c327ba None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:58.603503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa15a1b-2b1a-4a3c-88b4-7f1416c327ba None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:58.603866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa15a1b-2b1a-4a3c-88b4-7f1416c327ba None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:58.604541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa15a1b-2b1a-4a3c-88b4-7f1416c327ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:58.606010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa15a1b-2b1a-4a3c-88b4-7f1416c327ba None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:58.625803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 439/880] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:17:58 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:17:58.690490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6919b-1c81-4788-988e-3b541b57af5b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:58.738480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a85940-0404-4dbb-9c9d-8a6a28de5e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:58.739015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a85940-0404-4dbb-9c9d-8a6a28de5e01 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:58.739285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a85940-0404-4dbb-9c9d-8a6a28de5e01 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:58.739607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a85940-0404-4dbb-9c9d-8a6a28de5e01 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:58.740333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a85940-0404-4dbb-9c9d-8a6a28de5e01 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:58.740661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a85940-0404-4dbb-9c9d-8a6a28de5e01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:58.742183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a85940-0404-4dbb-9c9d-8a6a28de5e01 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:58.747863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 442/881] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:17:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:58.820300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d1f49-97a1-4197-bc02-fc17526c93ce None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:58.820620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d1f49-97a1-4197-bc02-fc17526c93ce None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:58.820891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d1f49-97a1-4197-bc02-fc17526c93ce None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:58.822488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 440/882] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:58.827974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db60f7c-671b-4bea-bdaf-185b1e64ff89 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:58.828254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db60f7c-671b-4bea-bdaf-185b1e64ff89 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:58.828526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db60f7c-671b-4bea-bdaf-185b1e64ff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:58.849026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0db60f7c-671b-4bea-bdaf-185b1e64ff89 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:58.850879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0db60f7c-671b-4bea-bdaf-185b1e64ff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:58.906209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:58.932689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 443/883] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:59.112048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b6ef3-fba1-49a3-a8fc-cc121bed36ed None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:59.112616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b6ef3-fba1-49a3-a8fc-cc121bed36ed None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:59.112616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b6ef3-fba1-49a3-a8fc-cc121bed36ed None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:59.114211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 441/884] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:17:59.119330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa4f87-aea7-44eb-828b-72e6f0671b95 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:59.119619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa4f87-aea7-44eb-828b-72e6f0671b95 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:59.119897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa4f87-aea7-44eb-828b-72e6f0671b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:59.138438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5efa4f87-aea7-44eb-828b-72e6f0671b95 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:17:59.139634 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5efa4f87-aea7-44eb-828b-72e6f0671b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:17:59.203236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b6ef3-fba1-49a3-a8fc-cc121bed36ed None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:59.212380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 444/885] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:17:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:17:59.256387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:59.318880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b6ef3-fba1-49a3-a8fc-cc121bed36ed None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:59.354864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4c080-490f-4684-8bfe-af4cef529910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:59.355487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4c080-490f-4684-8bfe-af4cef529910 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:59.355760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4c080-490f-4684-8bfe-af4cef529910 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:59.356027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4c080-490f-4684-8bfe-af4cef529910 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:59.367024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4c080-490f-4684-8bfe-af4cef529910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-89a4c080-490f-4684-8bfe-af4cef529910 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:17:59.372515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:59.373450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:59.374490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 442/886] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 00:17:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:59.385629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa4f87-aea7-44eb-828b-72e6f0671b95 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:59.487441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4c080-490f-4684-8bfe-af4cef529910 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:59.491112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa4f87-aea7-44eb-828b-72e6f0671b95 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:59.504861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:59.531529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfaf33-e52d-4baa-ba57-9616d0c52d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:59.533223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfaf33-e52d-4baa-ba57-9616d0c52d6e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:59.533705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfaf33-e52d-4baa-ba57-9616d0c52d6e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:59.534170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfaf33-e52d-4baa-ba57-9616d0c52d6e None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:59.535024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfaf33-e52d-4baa-ba57-9616d0c52d6e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:59.535553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfaf33-e52d-4baa-ba57-9616d0c52d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:59.537152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfaf33-e52d-4baa-ba57-9616d0c52d6e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:59.546472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 445/887] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 00:17:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:59.552326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfaf33-e52d-4baa-ba57-9616d0c52d6e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:59.608571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4c080-490f-4684-8bfe-af4cef529910 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:59.610352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:59.645699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c259f87-065c-46d2-933b-043aa7b7a229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:59.646355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c259f87-065c-46d2-933b-043aa7b7a229 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:59.646696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c259f87-065c-46d2-933b-043aa7b7a229 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:59.647040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c259f87-065c-46d2-933b-043aa7b7a229 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:59.647692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c259f87-065c-46d2-933b-043aa7b7a229 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:59.649176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c259f87-065c-46d2-933b-043aa7b7a229 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-9c259f87-065c-46d2-933b-043aa7b7a229 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:59.658883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:59.659861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:17:59.665276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 443/888] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:59.666646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c259f87-065c-46d2-933b-043aa7b7a229 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:59.679876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfaf33-e52d-4baa-ba57-9616d0c52d6e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:59.717791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5f0b8-4f36-411d-a126-5ab179f1a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:59.718880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5f0b8-4f36-411d-a126-5ab179f1a08f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:59.719358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5f0b8-4f36-411d-a126-5ab179f1a08f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:59.719892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5f0b8-4f36-411d-a126-5ab179f1a08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:59.732217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5f0b8-4f36-411d-a126-5ab179f1a08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:59.734021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5f0b8-4f36-411d-a126-5ab179f1a08f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-7cb5f0b8-4f36-411d-a126-5ab179f1a08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:17:59.750765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:17:59.751991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:17:59.753109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:17:59.753109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:17:59.753109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:17:59.759908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 446/889] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:17:59.761445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5f0b8-4f36-411d-a126-5ab179f1a08f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:59.786692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c259f87-065c-46d2-933b-043aa7b7a229 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:59.795814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:59.826928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50507f-ac2d-4910-a5c1-5d824acd84ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:59.828018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50507f-ac2d-4910-a5c1-5d824acd84ed None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:59.828613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50507f-ac2d-4910-a5c1-5d824acd84ed None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:59.829107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50507f-ac2d-4910-a5c1-5d824acd84ed None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:59.830601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50507f-ac2d-4910-a5c1-5d824acd84ed None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:59.831158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50507f-ac2d-4910-a5c1-5d824acd84ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:59.833059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50507f-ac2d-4910-a5c1-5d824acd84ed None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:59.839653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 444/890] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:17:59.845572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50507f-ac2d-4910-a5c1-5d824acd84ed None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:17:59.880703 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:17:59.907696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5f0b8-4f36-411d-a126-5ab179f1a08f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:59.957435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbe1be-9a86-4404-bef7-25915d37ca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:17:59.958820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbe1be-9a86-4404-bef7-25915d37ca2d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:17:59.959485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbe1be-9a86-4404-bef7-25915d37ca2d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:17:59.959905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbe1be-9a86-4404-bef7-25915d37ca2d None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:17:59.960716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbe1be-9a86-4404-bef7-25915d37ca2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:17:59.961179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbe1be-9a86-4404-bef7-25915d37ca2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:17:59.961548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50507f-ac2d-4910-a5c1-5d824acd84ed None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:17:59.962864 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbe1be-9a86-4404-bef7-25915d37ca2d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:17:59.984471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 447/891] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:17:59.990914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbe1be-9a86-4404-bef7-25915d37ca2d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:00.005591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68a8c1-7f08-4325-94f8-c7ed40153f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:00.006204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68a8c1-7f08-4325-94f8-c7ed40153f76 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.006482 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68a8c1-7f08-4325-94f8-c7ed40153f76 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.006754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68a8c1-7f08-4325-94f8-c7ed40153f76 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.007383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68a8c1-7f08-4325-94f8-c7ed40153f76 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:00.007682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68a8c1-7f08-4325-94f8-c7ed40153f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:00.009185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68a8c1-7f08-4325-94f8-c7ed40153f76 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:00.019360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 445/892] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:17:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:00.020806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68a8c1-7f08-4325-94f8-c7ed40153f76 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:00.109363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:00.128217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:00.134247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbe1be-9a86-4404-bef7-25915d37ca2d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:00.136866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68a8c1-7f08-4325-94f8-c7ed40153f76 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:00.175418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8190bea-014a-448d-bde7-39d9c4622793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:00.176062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8190bea-014a-448d-bde7-39d9c4622793 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.176404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8190bea-014a-448d-bde7-39d9c4622793 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.176814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8190bea-014a-448d-bde7-39d9c4622793 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.177782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8190bea-014a-448d-bde7-39d9c4622793 None admin] RBAC: Adding query filter params (interface=public, service_id=edecbbfda3ac45a2bc2338481665f87b, region_id=RegionOne) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:00.178155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8190bea-014a-448d-bde7-39d9c4622793 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:00.179070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079e47b-d708-4325-adb3-f184e72d107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:00.179816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8190bea-014a-448d-bde7-39d9c4622793 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:00.180101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079e47b-d708-4325-adb3-f184e72d107a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.180574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079e47b-d708-4325-adb3-f184e72d107a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.181017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079e47b-d708-4325-adb3-f184e72d107a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.190120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079e47b-d708-4325-adb3-f184e72d107a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:00.190814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 446/893] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 00:18:00 2022] GET /identity/v3/endpoints?service_id=edecbbfda3ac45a2bc2338481665f87b&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:00.192113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079e47b-d708-4325-adb3-f184e72d107a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:00.192233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8190bea-014a-448d-bde7-39d9c4622793 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-9079e47b-d708-4325-adb3-f184e72d107a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:18:00.206990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:00.207963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:18:00.208927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:00.215037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 448/894] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:17:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:00.216436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079e47b-d708-4325-adb3-f184e72d107a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:00.329271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8190bea-014a-448d-bde7-39d9c4622793 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:00.329834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079e47b-d708-4325-adb3-f184e72d107a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:00.367763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906fc08-afcd-4406-b88b-2f1916bbe437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:00.368374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906fc08-afcd-4406-b88b-2f1916bbe437 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.368686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906fc08-afcd-4406-b88b-2f1916bbe437 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.368966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906fc08-afcd-4406-b88b-2f1916bbe437 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.369757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906fc08-afcd-4406-b88b-2f1916bbe437 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:00.370002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906fc08-afcd-4406-b88b-2f1916bbe437 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:00.371646 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906fc08-afcd-4406-b88b-2f1916bbe437 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:00.374457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997db1da-a733-411c-8ea4-12fd054ab52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:00.376062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997db1da-a733-411c-8ea4-12fd054ab52a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.377101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997db1da-a733-411c-8ea4-12fd054ab52a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.377595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997db1da-a733-411c-8ea4-12fd054ab52a None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.378419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997db1da-a733-411c-8ea4-12fd054ab52a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:00.380162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997db1da-a733-411c-8ea4-12fd054ab52a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:00.387819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 447/895] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 00:18:00 2022] GET /identity/v3/services => generated 1268 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:18:00.396527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 449/896] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 00:18:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:00.403718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997db1da-a733-411c-8ea4-12fd054ab52a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:00.456992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:00.520387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997db1da-a733-411c-8ea4-12fd054ab52a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:00.558946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fe1f6-72d0-4977-8fc9-343eb6d0dc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:00.559556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fe1f6-72d0-4977-8fc9-343eb6d0dc2f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.559832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fe1f6-72d0-4977-8fc9-343eb6d0dc2f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.560103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fe1f6-72d0-4977-8fc9-343eb6d0dc2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.633912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fe1f6-72d0-4977-8fc9-343eb6d0dc2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:00.635376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fe1f6-72d0-4977-8fc9-343eb6d0dc2f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:00.660308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:00.679720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818aed70-544f-421f-a1b5-1af37d013940 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.680007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818aed70-544f-421f-a1b5-1af37d013940 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.680279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818aed70-544f-421f-a1b5-1af37d013940 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.681980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 450/897] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:00.686986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec03ddb-9ac4-462b-8b5f-4b92bd0e2589 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.687247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec03ddb-9ac4-462b-8b5f-4b92bd0e2589 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.687528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec03ddb-9ac4-462b-8b5f-4b92bd0e2589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.694471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:00.708981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bec03ddb-9ac4-462b-8b5f-4b92bd0e2589 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:00.710548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bec03ddb-9ac4-462b-8b5f-4b92bd0e2589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:00.745596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 448/898] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:00 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:18:00.751146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fe1f6-72d0-4977-8fc9-343eb6d0dc2f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:00.785542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 451/899] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:00.787180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81503a04-d7b8-4899-b3be-8a03891e1b7a None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.787667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81503a04-d7b8-4899-b3be-8a03891e1b7a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.788141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81503a04-d7b8-4899-b3be-8a03891e1b7a None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.789614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 452/900] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:00.794563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536e7a2-b990-435e-8060-665b2851c08f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.795141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536e7a2-b990-435e-8060-665b2851c08f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.795611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536e7a2-b990-435e-8060-665b2851c08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.816158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f536e7a2-b990-435e-8060-665b2851c08f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:00.817741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f536e7a2-b990-435e-8060-665b2851c08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:00.854311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fe1f6-72d0-4977-8fc9-343eb6d0dc2f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:00.878121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:00.890307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ce5bd-1899-45ff-ab22-e5ecf6615b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:00.890936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce5bd-1899-45ff-ab22-e5ecf6615b2d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:00.891207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce5bd-1899-45ff-ab22-e5ecf6615b2d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:00.891480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce5bd-1899-45ff-ab22-e5ecf6615b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:00.898418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 453/901] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:00.908557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ce5bd-1899-45ff-ab22-e5ecf6615b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:00.908557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ce5bd-1899-45ff-ab22-e5ecf6615b2d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-371ce5bd-1899-45ff-ab22-e5ecf6615b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:00.919528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:00.920519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:00.921413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:00.921413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:00.921413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:00.921413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 449/902] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:00.926757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536e7a2-b990-435e-8060-665b2851c08f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:00.989518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:01.030823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536e7a2-b990-435e-8060-665b2851c08f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.065553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878084d-84fb-4806-bec6-6b6e89cb33bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:01.066151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878084d-84fb-4806-bec6-6b6e89cb33bb None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:01.066425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878084d-84fb-4806-bec6-6b6e89cb33bb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:01.066692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878084d-84fb-4806-bec6-6b6e89cb33bb None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:01.067330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878084d-84fb-4806-bec6-6b6e89cb33bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:01.067625 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878084d-84fb-4806-bec6-6b6e89cb33bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:01.069180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878084d-84fb-4806-bec6-6b6e89cb33bb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:01.081773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 454/903] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:01.087274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ce5bd-1899-45ff-ab22-e5ecf6615b2d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:01.090294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878084d-84fb-4806-bec6-6b6e89cb33bb None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:01.200099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878084d-84fb-4806-bec6-6b6e89cb33bb None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.205261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ce5bd-1899-45ff-ab22-e5ecf6615b2d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.224776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:01.239877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24758d-ca68-4893-834e-5715a5cf803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:01.240449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24758d-ca68-4893-834e-5715a5cf803f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:01.240772 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24758d-ca68-4893-834e-5715a5cf803f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:01.241100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24758d-ca68-4893-834e-5715a5cf803f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:01.242219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bd429-ecb1-453d-bfd1-a6ae11231bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:01.242849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bd429-ecb1-453d-bfd1-a6ae11231bca None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:01.243123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bd429-ecb1-453d-bfd1-a6ae11231bca None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:01.243390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bd429-ecb1-453d-bfd1-a6ae11231bca None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:01.243965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bd429-ecb1-453d-bfd1-a6ae11231bca None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:01.245365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bd429-ecb1-453d-bfd1-a6ae11231bca None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:01.248721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24758d-ca68-4893-834e-5715a5cf803f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:01.250314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24758d-ca68-4893-834e-5715a5cf803f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-560bd429-ecb1-453d-bfd1-a6ae11231bca None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:01.258138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:01.259034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:01.259933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 455/904] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 1 00:18:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:01.260746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bd429-ecb1-453d-bfd1-a6ae11231bca None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-3e24758d-ca68-4893-834e-5715a5cf803f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:18:01.267317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:01.268186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 450/905] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:01.275041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24758d-ca68-4893-834e-5715a5cf803f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:01.317184 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:01.371291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bd429-ecb1-453d-bfd1-a6ae11231bca None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.396854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24758d-ca68-4893-834e-5715a5cf803f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.405693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf241c-5ac7-40d7-9955-fe470e69f847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:01.406300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf241c-5ac7-40d7-9955-fe470e69f847 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:01.406574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf241c-5ac7-40d7-9955-fe470e69f847 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:01.406842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf241c-5ac7-40d7-9955-fe470e69f847 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:01.407559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf241c-5ac7-40d7-9955-fe470e69f847 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:01.409179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf241c-5ac7-40d7-9955-fe470e69f847 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:01.427361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 456/906] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:18:01 2022] POST /identity/v3/services => generated 241 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:01.429259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf241c-5ac7-40d7-9955-fe470e69f847 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:01.460941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d045791-1bd9-42cd-8e49-ff8d042ad6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:01.461573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d045791-1bd9-42cd-8e49-ff8d042ad6d4 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:01.461853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d045791-1bd9-42cd-8e49-ff8d042ad6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:01.462123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d045791-1bd9-42cd-8e49-ff8d042ad6d4 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:01.463742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d045791-1bd9-42cd-8e49-ff8d042ad6d4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:01.464051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d045791-1bd9-42cd-8e49-ff8d042ad6d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:01.465561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d045791-1bd9-42cd-8e49-ff8d042ad6d4 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:01.477301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 451/907] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:18:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:01.478192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d045791-1bd9-42cd-8e49-ff8d042ad6d4 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:01.492004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:01.557483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf241c-5ac7-40d7-9955-fe470e69f847 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.591160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:01.596994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6031e70-00be-4b83-b493-b9ad29d90f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:01.597983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031e70-00be-4b83-b493-b9ad29d90f58 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:01.598468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031e70-00be-4b83-b493-b9ad29d90f58 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:01.598936 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031e70-00be-4b83-b493-b9ad29d90f58 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:01.599913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6031e70-00be-4b83-b493-b9ad29d90f58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:01.600220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6031e70-00be-4b83-b493-b9ad29d90f58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:01.600729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d045791-1bd9-42cd-8e49-ff8d042ad6d4 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.601763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6031e70-00be-4b83-b493-b9ad29d90f58 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:01.614866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 457/908] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:18:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:01.620765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6031e70-00be-4b83-b493-b9ad29d90f58 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:01.641803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56bea2-4393-4dfe-b0c3-556cbdc6befe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:01.642455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56bea2-4393-4dfe-b0c3-556cbdc6befe None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:01.642733 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56bea2-4393-4dfe-b0c3-556cbdc6befe None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:01.643003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e56bea2-4393-4dfe-b0c3-556cbdc6befe None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:01.643635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56bea2-4393-4dfe-b0c3-556cbdc6befe None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:01.643941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56bea2-4393-4dfe-b0c3-556cbdc6befe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:01.645962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e56bea2-4393-4dfe-b0c3-556cbdc6befe None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:01.653207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 452/909] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:18:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:01.659941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56bea2-4393-4dfe-b0c3-556cbdc6befe None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:01.755866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6031e70-00be-4b83-b493-b9ad29d90f58 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.766703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:01.774435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e56bea2-4393-4dfe-b0c3-556cbdc6befe None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.807618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3e206-929e-4c8e-953b-5c3ff6d29db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:01.808373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e206-929e-4c8e-953b-5c3ff6d29db8 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:01.808784 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e206-929e-4c8e-953b-5c3ff6d29db8 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:01.809055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3e206-929e-4c8e-953b-5c3ff6d29db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:01.809781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e206-929e-4c8e-953b-5c3ff6d29db8 None admin] RBAC: Adding query filter params (role.id=448c7e73924b47a1b6af2135604b5dbb, scope.project.id=ed93469862874c75951d95e28929c904, user.id=6eb1b515c4a74ba5a9da1c94ec914a75) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:01.810087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e206-929e-4c8e-953b-5c3ff6d29db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:01.811672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3e206-929e-4c8e-953b-5c3ff6d29db8 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:01.817506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b28b5f-23ff-4833-903a-607257738c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:01.818458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b28b5f-23ff-4833-903a-607257738c2d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:01.818699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 458/910] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:01 2022] GET /identity/v3/role_assignments?user.id=6eb1b515c4a74ba5a9da1c94ec914a75&scope.project.id=ed93469862874c75951d95e28929c904&role.id=448c7e73924b47a1b6af2135604b5dbb => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:01.819142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b28b5f-23ff-4833-903a-607257738c2d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:01.819624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b28b5f-23ff-4833-903a-607257738c2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:01.820651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b28b5f-23ff-4833-903a-607257738c2d None admin] RBAC: Adding query filter params (interface=public, service_id=4f4ced53af344cd6b774e1d7ffc20d46, region_id=RegionOne) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:01.821181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b28b5f-23ff-4833-903a-607257738c2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:01.822793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b28b5f-23ff-4833-903a-607257738c2d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:01.833806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 453/911] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 00:18:01 2022] GET /identity/v3/endpoints?service_id=4f4ced53af344cd6b774e1d7ffc20d46&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:01.839002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3e206-929e-4c8e-953b-5c3ff6d29db8 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:01.959011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3e206-929e-4c8e-953b-5c3ff6d29db8 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:01.999615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc2f10-03d8-4f42-bb17-3be1798422f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:02.000222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc2f10-03d8-4f42-bb17-3be1798422f8 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:02.000541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc2f10-03d8-4f42-bb17-3be1798422f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:02.000795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc2f10-03d8-4f42-bb17-3be1798422f8 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:02.001464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc2f10-03d8-4f42-bb17-3be1798422f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:02.002934 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc2f10-03d8-4f42-bb17-3be1798422f8 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:02.009699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 459/912] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 00:18:01 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:18:02.411510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2830c-4c90-4d2f-9ea4-a469d231a9c8 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:02.412046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2830c-4c90-4d2f-9ea4-a469d231a9c8 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:02.412046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2830c-4c90-4d2f-9ea4-a469d231a9c8 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:02.413708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 454/913] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:02.418901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef904119-9c87-423c-ab6c-ea71f5761475 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:02.419193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef904119-9c87-423c-ab6c-ea71f5761475 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:02.419471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef904119-9c87-423c-ab6c-ea71f5761475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:02.437923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ef904119-9c87-423c-ab6c-ea71f5761475 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:02.439646 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ef904119-9c87-423c-ab6c-ea71f5761475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:02.468082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:02.513449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 460/914] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:02.781199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2830c-4c90-4d2f-9ea4-a469d231a9c8 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:02.806736 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:02.889200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2830c-4c90-4d2f-9ea4-a469d231a9c8 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:02.926536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789e0ad-a00e-4420-b713-cb81449f071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:02.927176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789e0ad-a00e-4420-b713-cb81449f071a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:02.927787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789e0ad-a00e-4420-b713-cb81449f071a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:02.928068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789e0ad-a00e-4420-b713-cb81449f071a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:02.928696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789e0ad-a00e-4420-b713-cb81449f071a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:02.930104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789e0ad-a00e-4420-b713-cb81449f071a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-9789e0ad-a00e-4420-b713-cb81449f071a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:02.939257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:02.940255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 00:18:02.941141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:02.944637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 455/915] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:18:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:02.946796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef904119-9c87-423c-ab6c-ea71f5761475 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:03.055631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef904119-9c87-423c-ab6c-ea71f5761475 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:03.093692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25cee2-39b2-4f0e-a2b7-b503e9ec8982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:03.095759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25cee2-39b2-4f0e-a2b7-b503e9ec8982 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:03.096255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25cee2-39b2-4f0e-a2b7-b503e9ec8982 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:03.096760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25cee2-39b2-4f0e-a2b7-b503e9ec8982 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:03.099422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25cee2-39b2-4f0e-a2b7-b503e9ec8982 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:03.100233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25cee2-39b2-4f0e-a2b7-b503e9ec8982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:03.101901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25cee2-39b2-4f0e-a2b7-b503e9ec8982 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:03.110269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 461/916] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:18:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:03.116021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789e0ad-a00e-4420-b713-cb81449f071a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:03.207266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:03.223001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789e0ad-a00e-4420-b713-cb81449f071a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:03.259044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db7ddf-e981-4da0-99af-083a45dd3f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:03.259656 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db7ddf-e981-4da0-99af-083a45dd3f01 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:03.259937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db7ddf-e981-4da0-99af-083a45dd3f01 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:03.260213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db7ddf-e981-4da0-99af-083a45dd3f01 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:03.260940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db7ddf-e981-4da0-99af-083a45dd3f01 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:03.261250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db7ddf-e981-4da0-99af-083a45dd3f01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:03.262633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db7ddf-e981-4da0-99af-083a45dd3f01 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:03.272525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 456/917] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 1 00:18:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:03.273847 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25cee2-39b2-4f0e-a2b7-b503e9ec8982 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:03.304687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:03.382874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25cee2-39b2-4f0e-a2b7-b503e9ec8982 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:03.418451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bc804-22c5-4a32-bac8-753b615fe174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:03.419058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bc804-22c5-4a32-bac8-753b615fe174 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:03.419335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bc804-22c5-4a32-bac8-753b615fe174 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:03.419603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bc804-22c5-4a32-bac8-753b615fe174 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:03.420275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bc804-22c5-4a32-bac8-753b615fe174 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:03.421672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bc804-22c5-4a32-bac8-753b615fe174 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:03.452663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 462/918] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:18:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:03.488476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a17189-95c5-48bc-90e9-aaebd47f98a3 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:03.489055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a17189-95c5-48bc-90e9-aaebd47f98a3 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:03.489055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a17189-95c5-48bc-90e9-aaebd47f98a3 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:03.490449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 457/919] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:03.496300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3986f-d614-466d-a328-6676fa77809d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:03.496751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3986f-d614-466d-a328-6676fa77809d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:03.497103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3986f-d614-466d-a328-6676fa77809d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:03.518001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3fb3986f-d614-466d-a328-6676fa77809d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:03.519245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3fb3986f-d614-466d-a328-6676fa77809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:03.604095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 463/920] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:03.878922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a17189-95c5-48bc-90e9-aaebd47f98a3 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:03.894208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2c9b7-51be-4abb-b438-0222cd9f0768 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:03.894890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2c9b7-51be-4abb-b438-0222cd9f0768 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:03.895360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2c9b7-51be-4abb-b438-0222cd9f0768 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:03.897323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 464/921] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:03.902594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f14ca3-c0de-4b61-8d14-cdb672e61868 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:03.903100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f14ca3-c0de-4b61-8d14-cdb672e61868 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:03.903568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f14ca3-c0de-4b61-8d14-cdb672e61868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:03.923433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-c8f14ca3-c0de-4b61-8d14-cdb672e61868 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:03.925368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c8f14ca3-c0de-4b61-8d14-cdb672e61868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:03.928662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:04.002916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 465/922] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:04.004274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b2a40-3745-463b-ab4a-a08a1529f51d None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.004810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b2a40-3745-463b-ab4a-a08a1529f51d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.004916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b2a40-3745-463b-ab4a-a08a1529f51d None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.006429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 466/923] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:04.010015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a17189-95c5-48bc-90e9-aaebd47f98a3 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.011336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fff9c-9b16-406f-b0c9-4aa14ee0dbce None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.011716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fff9c-9b16-406f-b0c9-4aa14ee0dbce None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.012260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fff9c-9b16-406f-b0c9-4aa14ee0dbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.037484 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-8d5fff9c-9b16-406f-b0c9-4aa14ee0dbce None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:04.038916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8d5fff9c-9b16-406f-b0c9-4aa14ee0dbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:04.047768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddc045-9085-478a-b340-a0f6e8e5b244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.048614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddc045-9085-478a-b340-a0f6e8e5b244 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.049166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddc045-9085-478a-b340-a0f6e8e5b244 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.049507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddc045-9085-478a-b340-a0f6e8e5b244 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.050174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ddc045-9085-478a-b340-a0f6e8e5b244 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:04.051743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ddc045-9085-478a-b340-a0f6e8e5b244 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-34ddc045-9085-478a-b340-a0f6e8e5b244 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:04.061319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:04.062409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:04.063257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 458/924] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:18:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:04.069279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddc045-9085-478a-b340-a0f6e8e5b244 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.090842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:04.117597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 467/925] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:04.184780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddc045-9085-478a-b340-a0f6e8e5b244 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.219322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac48fe-8238-4d27-856f-bb3ce16ffa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.220156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac48fe-8238-4d27-856f-bb3ce16ffa21 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.220627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac48fe-8238-4d27-856f-bb3ce16ffa21 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.221039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac48fe-8238-4d27-856f-bb3ce16ffa21 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.221819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac48fe-8238-4d27-856f-bb3ce16ffa21 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:04.222259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac48fe-8238-4d27-856f-bb3ce16ffa21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:04.223750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac48fe-8238-4d27-856f-bb3ce16ffa21 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:04.229302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 459/926] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:04.235980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fff9c-9b16-406f-b0c9-4aa14ee0dbce None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.287987 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac48fe-8238-4d27-856f-bb3ce16ffa21 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.314902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:04.354280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fff9c-9b16-406f-b0c9-4aa14ee0dbce None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.396631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac48fe-8238-4d27-856f-bb3ce16ffa21 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.398610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:04.399601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e689d-02e2-4d04-b521-85c825532667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.401573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e689d-02e2-4d04-b521-85c825532667 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.401811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e689d-02e2-4d04-b521-85c825532667 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.403432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e689d-02e2-4d04-b521-85c825532667 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.404088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e689d-02e2-4d04-b521-85c825532667 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:04.404391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e689d-02e2-4d04-b521-85c825532667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:04.405808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e689d-02e2-4d04-b521-85c825532667 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:04.414221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 468/927] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:04.414886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e689d-02e2-4d04-b521-85c825532667 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.442208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df782a-13d5-4165-b26f-85bfdcf5d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.442854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df782a-13d5-4165-b26f-85bfdcf5d1bc None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.443137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df782a-13d5-4165-b26f-85bfdcf5d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.443410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df782a-13d5-4165-b26f-85bfdcf5d1bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.454452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df782a-13d5-4165-b26f-85bfdcf5d1bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-10df782a-13d5-4165-b26f-85bfdcf5d1bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:04.459760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:04.464659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 460/928] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:04.465704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df782a-13d5-4165-b26f-85bfdcf5d1bc None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.531598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e689d-02e2-4d04-b521-85c825532667 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.570474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24f427-1779-49c4-9f04-c1e598303b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.570960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df782a-13d5-4165-b26f-85bfdcf5d1bc None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.571393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24f427-1779-49c4-9f04-c1e598303b4c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.571393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24f427-1779-49c4-9f04-c1e598303b4c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.571678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24f427-1779-49c4-9f04-c1e598303b4c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.572520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24f427-1779-49c4-9f04-c1e598303b4c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:04.574864 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24f427-1779-49c4-9f04-c1e598303b4c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:04.576793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-6f24f427-1779-49c4-9f04-c1e598303b4c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:04.588738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:04.589705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:04.590586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 469/929] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:04.591357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24f427-1779-49c4-9f04-c1e598303b4c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.612287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f112e-804d-41c3-b4aa-13df43eda786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.612927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f112e-804d-41c3-b4aa-13df43eda786 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.613209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f112e-804d-41c3-b4aa-13df43eda786 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.613528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f112e-804d-41c3-b4aa-13df43eda786 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.614234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f112e-804d-41c3-b4aa-13df43eda786 None admin] RBAC: Adding query filter params (interface=public, service_id=07df5fe82da840c1a8ce72dde6b4e685, region_id=RegionOne) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:04.614546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f112e-804d-41c3-b4aa-13df43eda786 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:04.615956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f112e-804d-41c3-b4aa-13df43eda786 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:04.621539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 461/930] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/endpoints?service_id=07df5fe82da840c1a8ce72dde6b4e685&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:04.622356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f112e-804d-41c3-b4aa-13df43eda786 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.625561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:04.708638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24f427-1779-49c4-9f04-c1e598303b4c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.730835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f112e-804d-41c3-b4aa-13df43eda786 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.745024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac90f45-4601-4761-b16f-c33112472b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.745630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac90f45-4601-4761-b16f-c33112472b0c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.745907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac90f45-4601-4761-b16f-c33112472b0c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.746179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac90f45-4601-4761-b16f-c33112472b0c None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.746836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac90f45-4601-4761-b16f-c33112472b0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:04.747141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac90f45-4601-4761-b16f-c33112472b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:04.748520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac90f45-4601-4761-b16f-c33112472b0c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:04.760559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 470/931] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:04.760559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac90f45-4601-4761-b16f-c33112472b0c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.771239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a87ac-fe5d-4178-b4c4-4743cf2f1fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.772166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a87ac-fe5d-4178-b4c4-4743cf2f1fa0 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.772452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a87ac-fe5d-4178-b4c4-4743cf2f1fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.772755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a87ac-fe5d-4178-b4c4-4743cf2f1fa0 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.773387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a87ac-fe5d-4178-b4c4-4743cf2f1fa0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:04.773695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a87ac-fe5d-4178-b4c4-4743cf2f1fa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:04.775060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565a87ac-fe5d-4178-b4c4-4743cf2f1fa0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:04.779779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 462/932] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:04.780585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a87ac-fe5d-4178-b4c4-4743cf2f1fa0 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.824540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:04.855323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:04.872526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac90f45-4601-4761-b16f-c33112472b0c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.892423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a87ac-fe5d-4178-b4c4-4743cf2f1fa0 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:04.910246 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690c86-dadd-406a-960f-9145f94086d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.910858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690c86-dadd-406a-960f-9145f94086d4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.911141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690c86-dadd-406a-960f-9145f94086d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.911412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690c86-dadd-406a-960f-9145f94086d4 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.912017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db690c86-dadd-406a-960f-9145f94086d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:04.913429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db690c86-dadd-406a-960f-9145f94086d4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:04.919429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 471/933] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:18:04.920215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690c86-dadd-406a-960f-9145f94086d4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:04.939707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d8281-6c25-445e-9b04-9530f0a6feb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:04.940851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d8281-6c25-445e-9b04-9530f0a6feb6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:04.941328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d8281-6c25-445e-9b04-9530f0a6feb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:04.941850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d8281-6c25-445e-9b04-9530f0a6feb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:04.955052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d8281-6c25-445e-9b04-9530f0a6feb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:04.957107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d8281-6c25-445e-9b04-9530f0a6feb6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-9d7d8281-6c25-445e-9b04-9530f0a6feb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:04.973929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:04.975250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:04.976134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:04.976134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:04.976134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:04.978142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 463/934] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:04.983744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d8281-6c25-445e-9b04-9530f0a6feb6 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:05.052066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690c86-dadd-406a-960f-9145f94086d4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:05.085972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:05.105244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d8281-6c25-445e-9b04-9530f0a6feb6 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:05.105835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2254f-c299-4746-8ba0-0166192a13d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:05.106459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2254f-c299-4746-8ba0-0166192a13d3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:05.106739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2254f-c299-4746-8ba0-0166192a13d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:05.107016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2254f-c299-4746-8ba0-0166192a13d3 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:05.107659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2254f-c299-4746-8ba0-0166192a13d3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:05.107991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2254f-c299-4746-8ba0-0166192a13d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:05.110991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2254f-c299-4746-8ba0-0166192a13d3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:05.117155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 472/935] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:05.122733 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2254f-c299-4746-8ba0-0166192a13d3 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:05.133971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:05.150493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008837c0-e763-48f7-8046-e159d881e045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:05.151608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008837c0-e763-48f7-8046-e159d881e045 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:05.152881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008837c0-e763-48f7-8046-e159d881e045 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:05.153426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008837c0-e763-48f7-8046-e159d881e045 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:05.154506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008837c0-e763-48f7-8046-e159d881e045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:05.154925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008837c0-e763-48f7-8046-e159d881e045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:05.156724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008837c0-e763-48f7-8046-e159d881e045 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:05.177691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 464/936] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:05.179818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008837c0-e763-48f7-8046-e159d881e045 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:05.244586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2254f-c299-4746-8ba0-0166192a13d3 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:05.283038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f900c-a9c4-40ee-8b57-e2ffb2b091bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:05.284217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f900c-a9c4-40ee-8b57-e2ffb2b091bb None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:05.284739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f900c-a9c4-40ee-8b57-e2ffb2b091bb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:05.285215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f900c-a9c4-40ee-8b57-e2ffb2b091bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:05.286183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f900c-a9c4-40ee-8b57-e2ffb2b091bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:05.287825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f900c-a9c4-40ee-8b57-e2ffb2b091bb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:05.295361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008837c0-e763-48f7-8046-e159d881e045 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:05.311033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 473/937] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:18:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:18:05.343900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b64ae-c61b-4bdf-93fb-ef189360d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:05.344586 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b64ae-c61b-4bdf-93fb-ef189360d3ac None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:05.345740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b64ae-c61b-4bdf-93fb-ef189360d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:05.345999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b64ae-c61b-4bdf-93fb-ef189360d3ac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:05.354533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b64ae-c61b-4bdf-93fb-ef189360d3ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:05.356029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b64ae-c61b-4bdf-93fb-ef189360d3ac None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-559b64ae-c61b-4bdf-93fb-ef189360d3ac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:18:05.370659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:05.371726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:05.372748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 465/938] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:18:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:05.381130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f900c-a9c4-40ee-8b57-e2ffb2b091bb None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:05.440274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:05.481879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa38df-fa0a-44ea-b482-b35c81579fde None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:05.482219 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa38df-fa0a-44ea-b482-b35c81579fde None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:05.482427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa38df-fa0a-44ea-b482-b35c81579fde None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:05.483760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 466/939] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:05.489342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60f045-afb1-42b6-9225-3c36a0039352 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:05.489518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60f045-afb1-42b6-9225-3c36a0039352 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:05.489764 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60f045-afb1-42b6-9225-3c36a0039352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:05.510072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f900c-a9c4-40ee-8b57-e2ffb2b091bb None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:05.513207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-bd60f045-afb1-42b6-9225-3c36a0039352 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:05.514391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-bd60f045-afb1-42b6-9225-3c36a0039352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:05.546716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b90c22-c3d6-4d3d-9eca-e7931b6419e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:05.547330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b90c22-c3d6-4d3d-9eca-e7931b6419e1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:05.547615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b90c22-c3d6-4d3d-9eca-e7931b6419e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:05.547925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b90c22-c3d6-4d3d-9eca-e7931b6419e1 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:05.548704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b90c22-c3d6-4d3d-9eca-e7931b6419e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:05.549014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b90c22-c3d6-4d3d-9eca-e7931b6419e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:05.550461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b90c22-c3d6-4d3d-9eca-e7931b6419e1 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:05.552012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:05.574395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 474/940] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 00:18:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:05.581621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b90c22-c3d6-4d3d-9eca-e7931b6419e1 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:05.607735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 467/941] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:05.699919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b90c22-c3d6-4d3d-9eca-e7931b6419e1 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:05.701469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:05.736346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33a879-0f3c-4072-ad4b-9737cbf65a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:05.737013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33a879-0f3c-4072-ad4b-9737cbf65a84 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:05.737288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33a879-0f3c-4072-ad4b-9737cbf65a84 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:05.737558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33a879-0f3c-4072-ad4b-9737cbf65a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:05.747883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33a879-0f3c-4072-ad4b-9737cbf65a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:05.749416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33a879-0f3c-4072-ad4b-9737cbf65a84 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-ac33a879-0f3c-4072-ad4b-9737cbf65a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:05.764299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:05.766105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:05.769096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:05.769096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:05.769096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:05.769096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 475/942] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:05.772346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60f045-afb1-42b6-9225-3c36a0039352 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:05.886112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60f045-afb1-42b6-9225-3c36a0039352 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:05.921547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33a879-0f3c-4072-ad4b-9737cbf65a84 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:05.922025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d459f-9735-4edc-b20e-19249e34bcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:05.923661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d459f-9735-4edc-b20e-19249e34bcc2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:05.924164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d459f-9735-4edc-b20e-19249e34bcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:05.924686 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d459f-9735-4edc-b20e-19249e34bcc2 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:05.925729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d459f-9735-4edc-b20e-19249e34bcc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:05.926363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d459f-9735-4edc-b20e-19249e34bcc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:05.927864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d459f-9735-4edc-b20e-19249e34bcc2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:05.944565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 468/943] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:05.946781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d459f-9735-4edc-b20e-19249e34bcc2 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:05.973498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:06.047817 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33a879-0f3c-4072-ad4b-9737cbf65a84 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:06.063816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d459f-9735-4edc-b20e-19249e34bcc2 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:06.097372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:06.098909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:06.099353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:06.100283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:06.101142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:06.102749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:06.104325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb9762-4c73-420b-89d1-426bc78df7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:06.105135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb9762-4c73-420b-89d1-426bc78df7c0 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:06.105519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb9762-4c73-420b-89d1-426bc78df7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:06.105891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb9762-4c73-420b-89d1-426bc78df7c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:06.106309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING keystone.server.flask.common [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 00:18:06.118446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb9762-4c73-420b-89d1-426bc78df7c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:06.120058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb9762-4c73-420b-89d1-426bc78df7c0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-56eb9762-4c73-420b-89d1-426bc78df7c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:18:06.133792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:06.134996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:06.136104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:18:06.136104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:18:06.136104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:18:06.136104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:18:06.136104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:18:06.136104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:18:06.136104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:06.136104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:06.142836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.sql.core [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 00:18:06.142836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 00:18:06.142836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [parameters: {'id': '34d2bf73a4d04a1db2a941de672ffbcd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 00:18:06.142836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93105) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 00:18:06.143516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 469/944] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:06.146612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb9762-4c73-420b-89d1-426bc78df7c0 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 34d2bf73a4d04a1db2a941de672ffbcd.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 34d2bf73a4d04a1db2a941de672ffbcd. Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application conn.query(q) Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 00:18:06.148888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application result.read() Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 00:18:06.149674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application next(self.gen) Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application yield resource Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application next(self.gen) Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise self.value Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application session.commit() Sep 01 00:18:06.150664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.session.flush() Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application compat.raise_( Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 00:18:06.151581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 00:18:06.152460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application conn.query(q) Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application result.read() Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 00:18:06.154007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [parameters: {'id': '34d2bf73a4d04a1db2a941de672ffbcd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:06.154855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:06.155644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 34d2bf73a4d04a1db2a941de672ffbcd. Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:06.158262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 476/945] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 1 00:18:05 2022] POST /identity/v3/projects => generated 262 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:06.162935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:06.180932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:06.276322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:06.301380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf577d5-f1a9-42e6-9e2c-e572ac73038c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:06.302496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb9762-4c73-420b-89d1-426bc78df7c0 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:06.339076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f107375-9e7e-41ad-9019-db72a97e1cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:06.340158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f107375-9e7e-41ad-9019-db72a97e1cdf None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:06.340727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f107375-9e7e-41ad-9019-db72a97e1cdf None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:06.341205 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f107375-9e7e-41ad-9019-db72a97e1cdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:06.350301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f107375-9e7e-41ad-9019-db72a97e1cdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:06.352206 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f107375-9e7e-41ad-9019-db72a97e1cdf None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:06.352784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49b848-ef6b-4d69-b0e9-a4ca953af58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:06.353558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49b848-ef6b-4d69-b0e9-a4ca953af58e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:06.353839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49b848-ef6b-4d69-b0e9-a4ca953af58e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:06.354111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49b848-ef6b-4d69-b0e9-a4ca953af58e None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:06.355075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49b848-ef6b-4d69-b0e9-a4ca953af58e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:06.355403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49b848-ef6b-4d69-b0e9-a4ca953af58e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:06.357304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49b848-ef6b-4d69-b0e9-a4ca953af58e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-3f107375-9e7e-41ad-9019-db72a97e1cdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:18:06.365953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:06.366858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:06.367715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:18:06.367715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:18:06.367715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:18:06.367715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:18:06.367715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:18:06.367715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:18:06.367715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:06.367715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:06.374614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 470/946] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:06.374763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 477/947] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:06.377760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49b848-ef6b-4d69-b0e9-a4ca953af58e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:06.381592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f107375-9e7e-41ad-9019-db72a97e1cdf None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:06.471281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:06.505956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f107375-9e7e-41ad-9019-db72a97e1cdf None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:06.521220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49b848-ef6b-4d69-b0e9-a4ca953af58e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:06.542089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2d696-d96c-429d-bcb8-efc3a60ed515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:06.543112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2d696-d96c-429d-bcb8-efc3a60ed515 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:06.543592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2d696-d96c-429d-bcb8-efc3a60ed515 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:06.544182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2d696-d96c-429d-bcb8-efc3a60ed515 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/6eb1b515c4a74ba5a9da1c94ec914a75/roles/448c7e73924b47a1b6af2135604b5dbb` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:06.546752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2d696-d96c-429d-bcb8-efc3a60ed515 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=6eb1b515c4a74ba5a9da1c94ec914a75, role_id=448c7e73924b47a1b6af2135604b5dbb)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:06.549221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2d696-d96c-429d-bcb8-efc3a60ed515 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:06.561760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c18b4d-60c9-4d4e-af9e-005636d0a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:06.562324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c18b4d-60c9-4d4e-af9e-005636d0a00f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:06.562655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c18b4d-60c9-4d4e-af9e-005636d0a00f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:06.562928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c18b4d-60c9-4d4e-af9e-005636d0a00f None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:06.564693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c18b4d-60c9-4d4e-af9e-005636d0a00f None admin] RBAC: Adding query filter params (name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:06.565015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c18b4d-60c9-4d4e-af9e-005636d0a00f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:06.566712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c18b4d-60c9-4d4e-af9e-005636d0a00f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:06.568634 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 478/948] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:18:06 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/6eb1b515c4a74ba5a9da1c94ec914a75/roles/448c7e73924b47a1b6af2135604b5dbb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:18:06.600439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 471/949] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:18:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:06.934538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb2f93-de65-463d-80ee-12142f5b0618 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:06.934836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb2f93-de65-463d-80ee-12142f5b0618 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:06.935019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb2f93-de65-463d-80ee-12142f5b0618 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:06.939866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 479/950] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:06.945529 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cfcf6-c851-472c-a8d5-f4eabfbe791d None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:06.946022 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cfcf6-c851-472c-a8d5-f4eabfbe791d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:06.946106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cfcf6-c851-472c-a8d5-f4eabfbe791d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:06.954618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:06.965701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-496cfcf6-c851-472c-a8d5-f4eabfbe791d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:06.966903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-496cfcf6-c851-472c-a8d5-f4eabfbe791d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:07.037761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 472/951] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:07.310385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb2f93-de65-463d-80ee-12142f5b0618 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:07.400035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea780c-ddd5-42d8-8f2e-5ca947fa225e None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:07.400384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea780c-ddd5-42d8-8f2e-5ca947fa225e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:07.400644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea780c-ddd5-42d8-8f2e-5ca947fa225e None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:07.402287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 473/952] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:07.407302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26e3d6-5981-4b22-a943-87a3a618ba02 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:07.407605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26e3d6-5981-4b22-a943-87a3a618ba02 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:07.407853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26e3d6-5981-4b22-a943-87a3a618ba02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:07.426245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0d26e3d6-5981-4b22-a943-87a3a618ba02 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:07.426877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb2f93-de65-463d-80ee-12142f5b0618 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:07.427425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0d26e3d6-5981-4b22-a943-87a3a618ba02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:07.459347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:07.466715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7235f-0ffe-4fda-af8e-bd90fd1d70b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:07.467415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7235f-0ffe-4fda-af8e-bd90fd1d70b7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:07.467699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7235f-0ffe-4fda-af8e-bd90fd1d70b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:07.467971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7235f-0ffe-4fda-af8e-bd90fd1d70b7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:07.468959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7235f-0ffe-4fda-af8e-bd90fd1d70b7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:07.470998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7235f-0ffe-4fda-af8e-bd90fd1d70b7 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-b6b7235f-0ffe-4fda-af8e-bd90fd1d70b7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:07.483264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:07.484518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:07.486253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:07.487050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 480/953] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:18:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:07.492224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7235f-0ffe-4fda-af8e-bd90fd1d70b7 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:07.518666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 474/954] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:07.603802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7235f-0ffe-4fda-af8e-bd90fd1d70b7 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:07.640221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e658-b405-47e5-96ed-088eddce4862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:07.641116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e658-b405-47e5-96ed-088eddce4862 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:07.641629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e658-b405-47e5-96ed-088eddce4862 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:07.642054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e658-b405-47e5-96ed-088eddce4862 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:07.642840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78e658-b405-47e5-96ed-088eddce4862 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:07.643299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78e658-b405-47e5-96ed-088eddce4862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:07.644833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78e658-b405-47e5-96ed-088eddce4862 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:07.650289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 481/955] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:18:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:07.658840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26e3d6-5981-4b22-a943-87a3a618ba02 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:07.727984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:07.769188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26e3d6-5981-4b22-a943-87a3a618ba02 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:07.785332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e658-b405-47e5-96ed-088eddce4862 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:07.814728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b0070-e9c3-4097-8973-0eb88a581261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:07.816111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b0070-e9c3-4097-8973-0eb88a581261 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:07.816830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b0070-e9c3-4097-8973-0eb88a581261 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:07.818872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b0070-e9c3-4097-8973-0eb88a581261 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:07.820104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b0070-e9c3-4097-8973-0eb88a581261 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:07.820422 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b0070-e9c3-4097-8973-0eb88a581261 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:07.822201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b0070-e9c3-4097-8973-0eb88a581261 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:07.830271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 475/956] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:18:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:07.835527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b0070-e9c3-4097-8973-0eb88a581261 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:07.858853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:07.908364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e658-b405-47e5-96ed-088eddce4862 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:07.943890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d10ee-fdd9-49c0-90a8-2be121994d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:07.944540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d10ee-fdd9-49c0-90a8-2be121994d58 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:07.944824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d10ee-fdd9-49c0-90a8-2be121994d58 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:07.945094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d10ee-fdd9-49c0-90a8-2be121994d58 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:07.945829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d10ee-fdd9-49c0-90a8-2be121994d58 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:07.947311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d10ee-fdd9-49c0-90a8-2be121994d58 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:07.952373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b0070-e9c3-4097-8973-0eb88a581261 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-717d10ee-fdd9-49c0-90a8-2be121994d58 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:07.957421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:07.958679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 00:18:07.959546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:07.963457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 482/957] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:07.964445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d10ee-fdd9-49c0-90a8-2be121994d58 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:07.981394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:07.990440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12662d7-73e6-44fc-9a13-6261a7f9fb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:07.991047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12662d7-73e6-44fc-9a13-6261a7f9fb69 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:07.991321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12662d7-73e6-44fc-9a13-6261a7f9fb69 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:07.991589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12662d7-73e6-44fc-9a13-6261a7f9fb69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:07.992399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12662d7-73e6-44fc-9a13-6261a7f9fb69 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:07.993727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12662d7-73e6-44fc-9a13-6261a7f9fb69 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:08.015347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 476/958] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:18:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:18:08.082495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d10ee-fdd9-49c0-90a8-2be121994d58 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:08.109983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:08.131158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b579f47-ac2c-4175-a7d0-02fe0c1ab057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:08.131778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b579f47-ac2c-4175-a7d0-02fe0c1ab057 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:08.132053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b579f47-ac2c-4175-a7d0-02fe0c1ab057 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:08.132323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b579f47-ac2c-4175-a7d0-02fe0c1ab057 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:08.133000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b579f47-ac2c-4175-a7d0-02fe0c1ab057 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:08.133305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b579f47-ac2c-4175-a7d0-02fe0c1ab057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:08.134702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b579f47-ac2c-4175-a7d0-02fe0c1ab057 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:08.144635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 483/959] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:18:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:08.146480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12662d7-73e6-44fc-9a13-6261a7f9fb69 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:08.252961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12662d7-73e6-44fc-9a13-6261a7f9fb69 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:08.288833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91066f9b-7a42-4567-99e4-51c9a0cd5d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:08.289827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91066f9b-7a42-4567-99e4-51c9a0cd5d1e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:08.290331 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91066f9b-7a42-4567-99e4-51c9a0cd5d1e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:08.290819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91066f9b-7a42-4567-99e4-51c9a0cd5d1e None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:08.291655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91066f9b-7a42-4567-99e4-51c9a0cd5d1e None admin] RBAC: Adding query filter params (type=compute) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:08.292226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91066f9b-7a42-4567-99e4-51c9a0cd5d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:08.293817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91066f9b-7a42-4567-99e4-51c9a0cd5d1e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:08.299704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 477/960] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:18:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:08.585818 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41738a76-1653-4b9d-9c18-2baade81cf89 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:08.586370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41738a76-1653-4b9d-9c18-2baade81cf89 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:08.586370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41738a76-1653-4b9d-9c18-2baade81cf89 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:08.588087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 484/961] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:08.593400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef3c03-ba3c-4884-903f-5e152e243701 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:08.593692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef3c03-ba3c-4884-903f-5e152e243701 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:08.593969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef3c03-ba3c-4884-903f-5e152e243701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:08.613417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b9ef3c03-ba3c-4884-903f-5e152e243701 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:08.614621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b9ef3c03-ba3c-4884-903f-5e152e243701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:08.662817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:08.687845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 478/962] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:08.725502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0216f8d-326d-499e-850f-8b5660722337 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:08.725835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0216f8d-326d-499e-850f-8b5660722337 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:08.726055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0216f8d-326d-499e-850f-8b5660722337 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:08.727670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 485/963] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:08.732925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eddfe-b5d7-4b7d-8330-619c6c156e3a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:08.733213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eddfe-b5d7-4b7d-8330-619c6c156e3a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:08.733490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3eddfe-b5d7-4b7d-8330-619c6c156e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:08.752535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-8c3eddfe-b5d7-4b7d-8330-619c6c156e3a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:08.753722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8c3eddfe-b5d7-4b7d-8330-619c6c156e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:08.824616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 479/964] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:08.963790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0216f8d-326d-499e-850f-8b5660722337 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:09.069993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0216f8d-326d-499e-850f-8b5660722337 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:09.093171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3eddfe-b5d7-4b7d-8330-619c6c156e3a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:09.106929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9eaf3-cae7-4072-8b09-1e48ffaf3234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:09.107896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9eaf3-cae7-4072-8b09-1e48ffaf3234 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:09.108390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9eaf3-cae7-4072-8b09-1e48ffaf3234 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:09.108881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9eaf3-cae7-4072-8b09-1e48ffaf3234 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:09.119897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec9eaf3-cae7-4072-8b09-1e48ffaf3234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-5ec9eaf3-cae7-4072-8b09-1e48ffaf3234 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:09.124983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:09.126656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:09.127604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:09.130252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 486/965] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 00:18:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:09.140088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9eaf3-cae7-4072-8b09-1e48ffaf3234 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:09.198581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:09.213524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3eddfe-b5d7-4b7d-8330-619c6c156e3a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:09.234835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:09.270148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9eaf3-cae7-4072-8b09-1e48ffaf3234 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:09.275255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:09.276069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:09.276533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:09.276980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:09.277772 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:09.279962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:09.283577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WARNING keystone.server.flask.common [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 00:18:09.300253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.sql.core [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 00:18:09.300253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 00:18:09.300253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [parameters: {'user_id': '6bc2773f65c24816b7be09febe8afb6f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 00:18:09.300253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93106) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 00:18:09.307564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6946a-483e-4cab-8b14-a68aa65f87d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:09.308251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6946a-483e-4cab-8b14-a68aa65f87d6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:09.308658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6946a-483e-4cab-8b14-a68aa65f87d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:09.309026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6946a-483e-4cab-8b14-a68aa65f87d6 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:09.309761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6946a-483e-4cab-8b14-a68aa65f87d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:09.310135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6946a-483e-4cab-8b14-a68aa65f87d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:09.311591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6946a-483e-4cab-8b14-a68aa65f87d6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application conn.query(q) Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application result.read() Sep 01 00:18:09.320450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 00:18:09.321361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application next(self.gen) Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application yield resource Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application next(self.gen) Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise self.value Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application session.commit() Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 00:18:09.322316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self.session.flush() Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application compat.raise_( Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 00:18:09.323210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 00:18:09.324099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application conn.query(q) Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application result.read() Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 00:18:09.326376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [parameters: {'user_id': '6bc2773f65c24816b7be09febe8afb6f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:09.328435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 00:18:09.331050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 00:18:09.333607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 00:18:09.333607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 00:18:09.333607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 00:18:09.333607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 00:18:09.333607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 00:18:09.333607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:09.334459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 487/966] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 00:18:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:09.334595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 480/967] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 1 00:18:09 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:09.334723 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6946a-483e-4cab-8b14-a68aa65f87d6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:09.334877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:09.453434 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6946a-483e-4cab-8b14-a68aa65f87d6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:09.456320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:09.469546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cc3ae-2ed9-4ef5-bc96-9e20942c61e9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:09.492907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382ab7f-413e-4567-9505-774bbdc660e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:09.493888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382ab7f-413e-4567-9505-774bbdc660e4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:09.494374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382ab7f-413e-4567-9505-774bbdc660e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:09.494845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382ab7f-413e-4567-9505-774bbdc660e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:09.506697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382ab7f-413e-4567-9505-774bbdc660e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:09.508525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382ab7f-413e-4567-9505-774bbdc660e4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:09.516310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72f90a-d613-4027-81d0-575240f19444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:09.517882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72f90a-d613-4027-81d0-575240f19444 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:09.518371 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72f90a-d613-4027-81d0-575240f19444 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:09.519351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72f90a-d613-4027-81d0-575240f19444 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f382ab7f-413e-4567-9505-774bbdc660e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:09.526926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:09.527876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:09.528844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:09.528844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:09.528844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:09.530733 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72f90a-d613-4027-81d0-575240f19444 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:09.531176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 488/968] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:09.536544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72f90a-d613-4027-81d0-575240f19444 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:09.537416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382ab7f-413e-4567-9505-774bbdc660e4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-2c72f90a-d613-4027-81d0-575240f19444 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:18:09.550615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:09.551588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 00:18:09.552553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:09.555163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 481/969] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:18:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:09.560435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72f90a-d613-4027-81d0-575240f19444 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:09.605465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:09.660206 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382ab7f-413e-4567-9505-774bbdc660e4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:09.700064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72f90a-d613-4027-81d0-575240f19444 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:09.701874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431182a-8cf9-459c-a8b6-27d80e447914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:09.702894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431182a-8cf9-459c-a8b6-27d80e447914 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:09.703366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431182a-8cf9-459c-a8b6-27d80e447914 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:09.704372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431182a-8cf9-459c-a8b6-27d80e447914 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:09.705338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431182a-8cf9-459c-a8b6-27d80e447914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:09.705832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431182a-8cf9-459c-a8b6-27d80e447914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:09.707444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431182a-8cf9-459c-a8b6-27d80e447914 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:09.721359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 489/970] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:09.727505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431182a-8cf9-459c-a8b6-27d80e447914 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:09.747744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795caa-e3d2-443c-983d-02da0b4d6818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:09.751212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795caa-e3d2-443c-983d-02da0b4d6818 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:09.751709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795caa-e3d2-443c-983d-02da0b4d6818 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:09.752504 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:09.752871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795caa-e3d2-443c-983d-02da0b4d6818 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:09.753793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3795caa-e3d2-443c-983d-02da0b4d6818 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:09.757170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3795caa-e3d2-443c-983d-02da0b4d6818 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:09.759020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3795caa-e3d2-443c-983d-02da0b4d6818 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:09.812617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 482/971] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 1 00:18:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:09.852176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431182a-8cf9-459c-a8b6-27d80e447914 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:09.894589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebe67b-2eb9-459f-ac59-77444e5ed752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:09.895239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebe67b-2eb9-459f-ac59-77444e5ed752 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:09.895512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebe67b-2eb9-459f-ac59-77444e5ed752 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:09.895782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebe67b-2eb9-459f-ac59-77444e5ed752 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:09.904315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebe67b-2eb9-459f-ac59-77444e5ed752 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:09.905951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebe67b-2eb9-459f-ac59-77444e5ed752 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-a2ebe67b-2eb9-459f-ac59-77444e5ed752 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:18:09.919176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:09.920178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 00:18:09.921126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:09.924619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 490/972] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:18:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:09.926949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795caa-e3d2-443c-983d-02da0b4d6818 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:10.015580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9c322-1eea-462e-a6e5-ec7e02e77454 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.016204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9c322-1eea-462e-a6e5-ec7e02e77454 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.017124 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9c322-1eea-462e-a6e5-ec7e02e77454 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.018985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 491/973] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:10.024198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb46729-c60d-41c6-b1d9-938623054173 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.024585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb46729-c60d-41c6-b1d9-938623054173 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.024932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb46729-c60d-41c6-b1d9-938623054173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.028271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:10.045432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795caa-e3d2-443c-983d-02da0b4d6818 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:10.052322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-1cb46729-c60d-41c6-b1d9-938623054173 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:10.054540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1cb46729-c60d-41c6-b1d9-938623054173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:10.085582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94b94b-e31a-431a-91b1-2bd38a34abf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:10.086203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94b94b-e31a-431a-91b1-2bd38a34abf8 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.086483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94b94b-e31a-431a-91b1-2bd38a34abf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.086750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94b94b-e31a-431a-91b1-2bd38a34abf8 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.087501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94b94b-e31a-431a-91b1-2bd38a34abf8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:10.087801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94b94b-e31a-431a-91b1-2bd38a34abf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:10.089280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94b94b-e31a-431a-91b1-2bd38a34abf8 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:10.114568 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 483/974] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 1 00:18:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:10.115874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94b94b-e31a-431a-91b1-2bd38a34abf8 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:10.140940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 492/975] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:10.221198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94b94b-e31a-431a-91b1-2bd38a34abf8 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:10.261074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705203d1-75ba-4352-bdb0-580822d675bc None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.261363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705203d1-75ba-4352-bdb0-580822d675bc None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.261641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705203d1-75ba-4352-bdb0-580822d675bc None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.263110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 493/976] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:10.263974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be11112-404f-4bff-a348-666e94d05a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:10.264634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be11112-404f-4bff-a348-666e94d05a29 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.264912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be11112-404f-4bff-a348-666e94d05a29 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.265180 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be11112-404f-4bff-a348-666e94d05a29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.268224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63418a-be57-4995-a96d-2c7632e0056c None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.268551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63418a-be57-4995-a96d-2c7632e0056c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.268757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63418a-be57-4995-a96d-2c7632e0056c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.281284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:10.285321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:10.287912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-4b63418a-be57-4995-a96d-2c7632e0056c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:10.289352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4b63418a-be57-4995-a96d-2c7632e0056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:10.337968 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be11112-404f-4bff-a348-666e94d05a29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:10.339416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be11112-404f-4bff-a348-666e94d05a29 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:10.372811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 494/977] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:10.417089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63418a-be57-4995-a96d-2c7632e0056c None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:10.438302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 484/978] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:10 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:18:10.444082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be11112-404f-4bff-a348-666e94d05a29 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:10.496108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:10.522573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63418a-be57-4995-a96d-2c7632e0056c None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:10.553767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be11112-404f-4bff-a348-666e94d05a29 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:10.557254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf582d9-a377-4b03-9433-f4137eccad0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:10.557882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf582d9-a377-4b03-9433-f4137eccad0f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.558158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf582d9-a377-4b03-9433-f4137eccad0f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.558430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf582d9-a377-4b03-9433-f4137eccad0f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.559007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf582d9-a377-4b03-9433-f4137eccad0f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:10.560379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf582d9-a377-4b03-9433-f4137eccad0f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:10.564102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-ccf582d9-a377-4b03-9433-f4137eccad0f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:10.571629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:10.572771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:10.573617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:10.576524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 495/979] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:18:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:10.583089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf582d9-a377-4b03-9433-f4137eccad0f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:10.594755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5dc6e-ecb5-4be6-8385-567805e36a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:10.595361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5dc6e-ecb5-4be6-8385-567805e36a4b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.595953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5dc6e-ecb5-4be6-8385-567805e36a4b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.596227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5dc6e-ecb5-4be6-8385-567805e36a4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.608811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5dc6e-ecb5-4be6-8385-567805e36a4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:10.611161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5dc6e-ecb5-4be6-8385-567805e36a4b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-06e5dc6e-ecb5-4be6-8385-567805e36a4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:10.630495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:10.631578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:10.637498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:10.637498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:10.637498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:10.637498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 485/980] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:10.637498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5dc6e-ecb5-4be6-8385-567805e36a4b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:10.682725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf582d9-a377-4b03-9433-f4137eccad0f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:10.716586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05548665-71bc-4676-85a8-a20b278cc7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:10.717182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548665-71bc-4676-85a8-a20b278cc7a9 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.717456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548665-71bc-4676-85a8-a20b278cc7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.717727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05548665-71bc-4676-85a8-a20b278cc7a9 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.718343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05548665-71bc-4676-85a8-a20b278cc7a9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:10.718639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05548665-71bc-4676-85a8-a20b278cc7a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:10.720110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05548665-71bc-4676-85a8-a20b278cc7a9 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:10.724948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 496/981] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:18:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:10.725711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05548665-71bc-4676-85a8-a20b278cc7a9 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:10.730366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:10.734516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5dc6e-ecb5-4be6-8385-567805e36a4b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:10.769759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39730826-e338-4af4-87d6-6e36d9f32198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:10.770365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730826-e338-4af4-87d6-6e36d9f32198 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.770642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730826-e338-4af4-87d6-6e36d9f32198 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.770917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39730826-e338-4af4-87d6-6e36d9f32198 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.771570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39730826-e338-4af4-87d6-6e36d9f32198 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:10.773068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39730826-e338-4af4-87d6-6e36d9f32198 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:10.788177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 486/982] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:18:10 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:10.789014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39730826-e338-4af4-87d6-6e36d9f32198 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:10.791530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:10.837336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05548665-71bc-4676-85a8-a20b278cc7a9 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:10.886834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51d265-ea31-4210-a258-4c34b99a5e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:10.887456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51d265-ea31-4210-a258-4c34b99a5e4e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.887740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51d265-ea31-4210-a258-4c34b99a5e4e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.888021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51d265-ea31-4210-a258-4c34b99a5e4e None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.888689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51d265-ea31-4210-a258-4c34b99a5e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:10.888997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51d265-ea31-4210-a258-4c34b99a5e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:10.890408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51d265-ea31-4210-a258-4c34b99a5e4e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:10.904170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 497/983] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:10.910511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51d265-ea31-4210-a258-4c34b99a5e4e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:10.919573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39730826-e338-4af4-87d6-6e36d9f32198 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:10.962894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15285fa5-2a29-4e63-8e2a-651e803eb4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:10.963596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15285fa5-2a29-4e63-8e2a-651e803eb4eb None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:10.963936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15285fa5-2a29-4e63-8e2a-651e803eb4eb None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:10.964207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15285fa5-2a29-4e63-8e2a-651e803eb4eb None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:10.964864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15285fa5-2a29-4e63-8e2a-651e803eb4eb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:10.965163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15285fa5-2a29-4e63-8e2a-651e803eb4eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:10.966532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15285fa5-2a29-4e63-8e2a-651e803eb4eb None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:10.972157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 487/984] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:18:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:10.978403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15285fa5-2a29-4e63-8e2a-651e803eb4eb None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:11.005526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:11.027440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51d265-ea31-4210-a258-4c34b99a5e4e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:11.048815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:11.064227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2950f-9ba7-47b8-a5bc-f66c4211b80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:11.064964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2950f-9ba7-47b8-a5bc-f66c4211b80a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:11.065316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2950f-9ba7-47b8-a5bc-f66c4211b80a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:11.065652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2950f-9ba7-47b8-a5bc-f66c4211b80a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:11.072356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2950f-9ba7-47b8-a5bc-f66c4211b80a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:11.073890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2950f-9ba7-47b8-a5bc-f66c4211b80a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-91c2950f-9ba7-47b8-a5bc-f66c4211b80a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:18:11.085768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:11.086837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:11.088043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:18:11.088043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:18:11.088043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:18:11.088043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:18:11.088043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:18:11.088043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:18:11.088043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:11.088043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:11.089052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 498/985] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:11.090585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15285fa5-2a29-4e63-8e2a-651e803eb4eb None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:11.095051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2950f-9ba7-47b8-a5bc-f66c4211b80a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:11.127881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b28415-a267-4fcd-920f-4c4150792253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:11.128626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b28415-a267-4fcd-920f-4c4150792253 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:11.128914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b28415-a267-4fcd-920f-4c4150792253 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:11.129208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b28415-a267-4fcd-920f-4c4150792253 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:11.129915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b28415-a267-4fcd-920f-4c4150792253 None admin] RBAC: Adding query filter params (interface=admin, service_id=07df5fe82da840c1a8ce72dde6b4e685, region_id=RegionOne) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:11.130227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b28415-a267-4fcd-920f-4c4150792253 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:11.131600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b28415-a267-4fcd-920f-4c4150792253 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:11.136724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 488/986] 10.222.0.25 () {56 vars in 1249 bytes} [Thu Sep 1 00:18:10 2022] GET /identity/v3/endpoints?service_id=07df5fe82da840c1a8ce72dde6b4e685&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:11.142807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b28415-a267-4fcd-920f-4c4150792253 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:11.212853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2950f-9ba7-47b8-a5bc-f66c4211b80a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:11.237158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:11.254476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b28415-a267-4fcd-920f-4c4150792253 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:11.258590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab605206-9aa0-4c83-90a8-3ad1f8508e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:11.259294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab605206-9aa0-4c83-90a8-3ad1f8508e74 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:11.259637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab605206-9aa0-4c83-90a8-3ad1f8508e74 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:11.259975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab605206-9aa0-4c83-90a8-3ad1f8508e74 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:11.260776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab605206-9aa0-4c83-90a8-3ad1f8508e74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:11.261144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab605206-9aa0-4c83-90a8-3ad1f8508e74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:11.262586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab605206-9aa0-4c83-90a8-3ad1f8508e74 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:11.275560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 499/987] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:11.284603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab605206-9aa0-4c83-90a8-3ad1f8508e74 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:11.293611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2989b-d2e1-4947-b975-9c07849f1f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:11.294231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2989b-d2e1-4947-b975-9c07849f1f13 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:11.294512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2989b-d2e1-4947-b975-9c07849f1f13 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:11.294787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2989b-d2e1-4947-b975-9c07849f1f13 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:11.295720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2989b-d2e1-4947-b975-9c07849f1f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:11.297136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2989b-d2e1-4947-b975-9c07849f1f13 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:11.304039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 489/988] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 00:18:11 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:18:11.338658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:11.403561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab605206-9aa0-4c83-90a8-3ad1f8508e74 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:11.441262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94d4c-4fbb-4ebf-bf3a-a37c4929a6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:11.441877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94d4c-4fbb-4ebf-bf3a-a37c4929a6b2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:11.442156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94d4c-4fbb-4ebf-bf3a-a37c4929a6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:11.442430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94d4c-4fbb-4ebf-bf3a-a37c4929a6b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:11.443164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94d4c-4fbb-4ebf-bf3a-a37c4929a6b2 None admin] RBAC: Adding query filter params (role.id=448c7e73924b47a1b6af2135604b5dbb, scope.project.id=ed93469862874c75951d95e28929c904, user.id=6eb1b515c4a74ba5a9da1c94ec914a75) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:11.443472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94d4c-4fbb-4ebf-bf3a-a37c4929a6b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:11.444944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94d4c-4fbb-4ebf-bf3a-a37c4929a6b2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:11.452122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 500/989] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:11 2022] GET /identity/v3/role_assignments?user.id=6eb1b515c4a74ba5a9da1c94ec914a75&scope.project.id=ed93469862874c75951d95e28929c904&role.id=448c7e73924b47a1b6af2135604b5dbb => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:12.728743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a1e0f-1c71-4608-b49e-a0586543a809 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:12.729301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a1e0f-1c71-4608-b49e-a0586543a809 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:12.729301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a1e0f-1c71-4608-b49e-a0586543a809 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:12.730656 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 490/990] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:12.736654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2da52a-1259-4397-ab5a-18bcc846581b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:12.737041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2da52a-1259-4397-ab5a-18bcc846581b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:12.737219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2da52a-1259-4397-ab5a-18bcc846581b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:12.755709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5b2da52a-1259-4397-ab5a-18bcc846581b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:12.756920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5b2da52a-1259-4397-ab5a-18bcc846581b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:12.823807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 501/991] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:13.089102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a1e0f-1c71-4608-b49e-a0586543a809 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:13.195086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a1e0f-1c71-4608-b49e-a0586543a809 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:13.232322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c372888-0a03-4e49-960f-2db9095e67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:13.232982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c372888-0a03-4e49-960f-2db9095e67b8 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.233259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c372888-0a03-4e49-960f-2db9095e67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.233529 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c372888-0a03-4e49-960f-2db9095e67b8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.234362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c372888-0a03-4e49-960f-2db9095e67b8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:13.236041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c372888-0a03-4e49-960f-2db9095e67b8 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:13.245405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fb191-78b7-405c-8a6a-c3ac23911126 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-2c372888-0a03-4e49-960f-2db9095e67b8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:13.245551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:13.246652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 00:18:13.252257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:13.253305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fb191-78b7-405c-8a6a-c3ac23911126 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.253305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fb191-78b7-405c-8a6a-c3ac23911126 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.253305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 502/992] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:13.253535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 491/993] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:13.256112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fb191-78b7-405c-8a6a-c3ac23911126 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:13.256585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3c253-1f7f-468b-bdc4-fe9154c0830d None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.256873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3c253-1f7f-468b-bdc4-fe9154c0830d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.257147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3c253-1f7f-468b-bdc4-fe9154c0830d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.275198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ffa3c253-1f7f-468b-bdc4-fe9154c0830d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:13.276818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ffa3c253-1f7f-468b-bdc4-fe9154c0830d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:13.286837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:13.295790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:13.357310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 492/994] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:13.364933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fb191-78b7-405c-8a6a-c3ac23911126 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:13.377629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc2a8-ed4e-41d9-9d16-3c804b0f3fa1 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.377914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc2a8-ed4e-41d9-9d16-3c804b0f3fa1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.378181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bc2a8-ed4e-41d9-9d16-3c804b0f3fa1 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.379664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 493/995] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:13.384197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9acb62-2418-4113-9844-c0d47ac9fcb9 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.384526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9acb62-2418-4113-9844-c0d47ac9fcb9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.384793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9acb62-2418-4113-9844-c0d47ac9fcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.402717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95153ee7-6fdc-4046-aea6-d6d0c2e076d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:13.403025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-6f9acb62-2418-4113-9844-c0d47ac9fcb9 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:13.403324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95153ee7-6fdc-4046-aea6-d6d0c2e076d1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.403596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95153ee7-6fdc-4046-aea6-d6d0c2e076d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.403871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95153ee7-6fdc-4046-aea6-d6d0c2e076d1 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.404252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6f9acb62-2418-4113-9844-c0d47ac9fcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:13.404523 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95153ee7-6fdc-4046-aea6-d6d0c2e076d1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:13.404832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95153ee7-6fdc-4046-aea6-d6d0c2e076d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:13.406196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95153ee7-6fdc-4046-aea6-d6d0c2e076d1 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:13.415327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 503/996] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:18:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:13.421324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95153ee7-6fdc-4046-aea6-d6d0c2e076d1 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:13.481240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 494/997] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:13.527359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:13.531606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95153ee7-6fdc-4046-aea6-d6d0c2e076d1 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:13.570097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56e61e-41e1-4c2e-b9de-9d69ecafb326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:13.570699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56e61e-41e1-4c2e-b9de-9d69ecafb326 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.570980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56e61e-41e1-4c2e-b9de-9d69ecafb326 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.571254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56e61e-41e1-4c2e-b9de-9d69ecafb326 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.571883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56e61e-41e1-4c2e-b9de-9d69ecafb326 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:13.572185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56e61e-41e1-4c2e-b9de-9d69ecafb326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:13.573603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c56e61e-41e1-4c2e-b9de-9d69ecafb326 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:13.583717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 504/998] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:18:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:13.584990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9acb62-2418-4113-9844-c0d47ac9fcb9 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:13.621828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56e61e-41e1-4c2e-b9de-9d69ecafb326 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:13.663675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:13.689747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9acb62-2418-4113-9844-c0d47ac9fcb9 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:13.725865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c14d5f-47b6-4b30-b749-d3cbd7aad5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:13.726472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c14d5f-47b6-4b30-b749-d3cbd7aad5dd None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.726749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c14d5f-47b6-4b30-b749-d3cbd7aad5dd None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.727020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c14d5f-47b6-4b30-b749-d3cbd7aad5dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.727711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c14d5f-47b6-4b30-b749-d3cbd7aad5dd None admin] RBAC: Adding query filter params (interface=public, service_id=9c4a9e03fe1644869710f5fccb56b619, region_id=RegionOne) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:13.728014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c14d5f-47b6-4b30-b749-d3cbd7aad5dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:13.729402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c14d5f-47b6-4b30-b749-d3cbd7aad5dd None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:13.733940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c56e61e-41e1-4c2e-b9de-9d69ecafb326 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:13.734712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 495/999] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 00:18:13 2022] GET /identity/v3/endpoints?service_id=9c4a9e03fe1644869710f5fccb56b619&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:13.740090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c14d5f-47b6-4b30-b749-d3cbd7aad5dd None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:13.772105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea210ed4-0ada-4dea-b11d-63d974a3f19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:13.772771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea210ed4-0ada-4dea-b11d-63d974a3f19c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.773037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea210ed4-0ada-4dea-b11d-63d974a3f19c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.773305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea210ed4-0ada-4dea-b11d-63d974a3f19c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.773885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea210ed4-0ada-4dea-b11d-63d974a3f19c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:13.775271 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea210ed4-0ada-4dea-b11d-63d974a3f19c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-ea210ed4-0ada-4dea-b11d-63d974a3f19c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:13.784081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:13.785286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:13.786676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 505/1000] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 1 00:18:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:13.787512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea210ed4-0ada-4dea-b11d-63d974a3f19c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:13.814538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:13.844075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c14d5f-47b6-4b30-b749-d3cbd7aad5dd None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:13.877482 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31f732-5e41-4ba7-b0e1-0cafa3797ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:13.877998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31f732-5e41-4ba7-b0e1-0cafa3797ca9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.878271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31f732-5e41-4ba7-b0e1-0cafa3797ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.878552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31f732-5e41-4ba7-b0e1-0cafa3797ca9 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.879141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31f732-5e41-4ba7-b0e1-0cafa3797ca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:13.880514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31f732-5e41-4ba7-b0e1-0cafa3797ca9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:13.886108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea210ed4-0ada-4dea-b11d-63d974a3f19c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:13.886654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 496/1001] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 00:18:13 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:18:13.897067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31f732-5e41-4ba7-b0e1-0cafa3797ca9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:13.897067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:13.924937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3ebf8-d66d-4f0c-b032-758ee2eb990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:13.925579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3ebf8-d66d-4f0c-b032-758ee2eb990d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:13.925844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3ebf8-d66d-4f0c-b032-758ee2eb990d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:13.926105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3ebf8-d66d-4f0c-b032-758ee2eb990d None admin] PATH_INFO: `/v3/services/image` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:13.926692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3ebf8-d66d-4f0c-b032-758ee2eb990d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:13.928078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3ebf8-d66d-4f0c-b032-758ee2eb990d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-1bc3ebf8-d66d-4f0c-b032-758ee2eb990d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:13.937535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:13.939488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 00:18:13.940309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:13.944797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 506/1002] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 1 00:18:13 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:13.945706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3ebf8-d66d-4f0c-b032-758ee2eb990d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:14.013833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31f732-5e41-4ba7-b0e1-0cafa3797ca9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:14.054164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80c25-7595-46ff-852d-26ab0af0009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:14.054776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80c25-7595-46ff-852d-26ab0af0009f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:14.055052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80c25-7595-46ff-852d-26ab0af0009f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:14.055312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80c25-7595-46ff-852d-26ab0af0009f None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:14.055946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80c25-7595-46ff-852d-26ab0af0009f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:14.056245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80c25-7595-46ff-852d-26ab0af0009f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:14.057657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80c25-7595-46ff-852d-26ab0af0009f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:14.061189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:14.062693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 497/1003] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:18:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:14.068858 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80c25-7595-46ff-852d-26ab0af0009f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:14.069825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3ebf8-d66d-4f0c-b032-758ee2eb990d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:14.109403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef3e23-be0d-4c1b-9c25-ff7e6ddb6fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:14.110104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3e23-be0d-4c1b-9c25-ff7e6ddb6fcc None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:14.110483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3e23-be0d-4c1b-9c25-ff7e6ddb6fcc None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:14.110910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef3e23-be0d-4c1b-9c25-ff7e6ddb6fcc None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:14.111536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef3e23-be0d-4c1b-9c25-ff7e6ddb6fcc None admin] RBAC: Adding query filter params (name=image) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:14.112076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef3e23-be0d-4c1b-9c25-ff7e6ddb6fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:14.113221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef3e23-be0d-4c1b-9c25-ff7e6ddb6fcc None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:14.118581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 507/1004] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:14.123819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef3e23-be0d-4c1b-9c25-ff7e6ddb6fcc None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:14.138478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:14.185169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80c25-7595-46ff-852d-26ab0af0009f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:14.221623 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e941c8a-5c30-4354-acc7-f4f026100908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:14.222239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e941c8a-5c30-4354-acc7-f4f026100908 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:14.222514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e941c8a-5c30-4354-acc7-f4f026100908 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:14.222783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e941c8a-5c30-4354-acc7-f4f026100908 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:14.223415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e941c8a-5c30-4354-acc7-f4f026100908 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:14.223714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e941c8a-5c30-4354-acc7-f4f026100908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:14.225412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e941c8a-5c30-4354-acc7-f4f026100908 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:14.230859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 498/1005] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 1 00:18:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:14.237096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e941c8a-5c30-4354-acc7-f4f026100908 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:14.246924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef3e23-be0d-4c1b-9c25-ff7e6ddb6fcc None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:14.285803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa9a88-a3c3-440c-9586-638edf57bf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:14.286442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa9a88-a3c3-440c-9586-638edf57bf4b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:14.286720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa9a88-a3c3-440c-9586-638edf57bf4b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:14.286994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa9a88-a3c3-440c-9586-638edf57bf4b None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:14.287620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa9a88-a3c3-440c-9586-638edf57bf4b None admin] RBAC: Adding query filter params (type=image) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:14.287921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa9a88-a3c3-440c-9586-638edf57bf4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:14.289335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa9a88-a3c3-440c-9586-638edf57bf4b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:14.294255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 508/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:14.361669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e941c8a-5c30-4354-acc7-f4f026100908 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:14.392997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:14.400343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3e0d1-f33e-4405-839c-e48cbc7788a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:14.401007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3e0d1-f33e-4405-839c-e48cbc7788a3 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:14.401287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3e0d1-f33e-4405-839c-e48cbc7788a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:14.401559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3e0d1-f33e-4405-839c-e48cbc7788a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:14.402236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3e0d1-f33e-4405-839c-e48cbc7788a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:14.403645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3e0d1-f33e-4405-839c-e48cbc7788a3 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:14.421856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 499/1007] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:18:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:15.805994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bc635-60f0-427c-a3d5-fb13895473f7 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:15.806550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bc635-60f0-427c-a3d5-fb13895473f7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:15.806550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bc635-60f0-427c-a3d5-fb13895473f7 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:15.808153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 509/1008] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:15.813839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11270055-ff93-4a79-a97e-cfb8111fa071 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:15.814273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11270055-ff93-4a79-a97e-cfb8111fa071 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:15.814365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11270055-ff93-4a79-a97e-cfb8111fa071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:15.834099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-11270055-ff93-4a79-a97e-cfb8111fa071 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:15.835294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-11270055-ff93-4a79-a97e-cfb8111fa071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:15.913487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 500/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:16.180876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bc635-60f0-427c-a3d5-fb13895473f7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:16.201020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:16.255533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3bef2-735a-4506-8e27-dc05e53ea3fa None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.255825 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3bef2-735a-4506-8e27-dc05e53ea3fa None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.256102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3bef2-735a-4506-8e27-dc05e53ea3fa None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.257512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 501/1010] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:16.262259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fea0a6-76f8-45a9-9db0-8d6deaa04a6d None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.262547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fea0a6-76f8-45a9-9db0-8d6deaa04a6d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.262829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fea0a6-76f8-45a9-9db0-8d6deaa04a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.281380 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-57fea0a6-76f8-45a9-9db0-8d6deaa04a6d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:16.283005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-57fea0a6-76f8-45a9-9db0-8d6deaa04a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:16.301793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669bc635-60f0-427c-a3d5-fb13895473f7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:16.339075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed784a-470e-44e7-81ed-d2331c1c3256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:16.339705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed784a-470e-44e7-81ed-d2331c1c3256 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.339988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed784a-470e-44e7-81ed-d2331c1c3256 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.340262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed784a-470e-44e7-81ed-d2331c1c3256 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.340880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed784a-470e-44e7-81ed-d2331c1c3256 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:16.342256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed784a-470e-44e7-81ed-d2331c1c3256 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:16.351577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 502/1011] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-4eed784a-470e-44e7-81ed-d2331c1c3256 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:16.351706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:16.353174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 00:18:16.354331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:16.355167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc7ad8-623d-4d91-8918-85862e0bdc39 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.355241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 510/1012] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:16.355308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc7ad8-623d-4d91-8918-85862e0bdc39 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.355308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc7ad8-623d-4d91-8918-85862e0bdc39 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.355308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 503/1013] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:16.360808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1dfd1-44cf-49ba-834b-ab707cabf56b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.361097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1dfd1-44cf-49ba-834b-ab707cabf56b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.361366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1dfd1-44cf-49ba-834b-ab707cabf56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.362787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc7ad8-623d-4d91-8918-85862e0bdc39 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:16.380297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-04c1dfd1-44cf-49ba-834b-ab707cabf56b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:16.381530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-04c1dfd1-44cf-49ba-834b-ab707cabf56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:16.416946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:16.454806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 511/1014] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:16.481367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc7ad8-623d-4d91-8918-85862e0bdc39 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:16.520693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf366c0-f5f5-4cd6-adfe-3dd19846ab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:16.521296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf366c0-f5f5-4cd6-adfe-3dd19846ab50 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.521571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf366c0-f5f5-4cd6-adfe-3dd19846ab50 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.521842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf366c0-f5f5-4cd6-adfe-3dd19846ab50 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.522464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf366c0-f5f5-4cd6-adfe-3dd19846ab50 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:16.522763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf366c0-f5f5-4cd6-adfe-3dd19846ab50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:16.524129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf366c0-f5f5-4cd6-adfe-3dd19846ab50 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:16.534519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 504/1015] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:18:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:16.535554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1dfd1-44cf-49ba-834b-ab707cabf56b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:16.550801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:16.625284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf366c0-f5f5-4cd6-adfe-3dd19846ab50 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:16.642723 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1dfd1-44cf-49ba-834b-ab707cabf56b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:16.679099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d241b1-69c6-4a82-b5c5-3a95851000e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:16.679731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d241b1-69c6-4a82-b5c5-3a95851000e9 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.680084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d241b1-69c6-4a82-b5c5-3a95851000e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.680347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d241b1-69c6-4a82-b5c5-3a95851000e9 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.681105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d241b1-69c6-4a82-b5c5-3a95851000e9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:16.681398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d241b1-69c6-4a82-b5c5-3a95851000e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:16.682774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d241b1-69c6-4a82-b5c5-3a95851000e9 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:16.689794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 512/1016] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 1 00:18:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:16.695122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d241b1-69c6-4a82-b5c5-3a95851000e9 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:16.726348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf366c0-f5f5-4cd6-adfe-3dd19846ab50 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:16.736640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:16.761268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e96121-c9c8-479b-a2d7-7352d1ab810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:16.761875 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e96121-c9c8-479b-a2d7-7352d1ab810a None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.762156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e96121-c9c8-479b-a2d7-7352d1ab810a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.762425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e96121-c9c8-479b-a2d7-7352d1ab810a None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.763145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e96121-c9c8-479b-a2d7-7352d1ab810a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:16.764648 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e96121-c9c8-479b-a2d7-7352d1ab810a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:16.776804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:16.784566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 505/1017] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:18:16 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:18:16.784566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e96121-c9c8-479b-a2d7-7352d1ab810a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:16.804454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d241b1-69c6-4a82-b5c5-3a95851000e9 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:16.851169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaf3e1-8bf0-40f3-a945-04ea377d52d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:16.851833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaf3e1-8bf0-40f3-a945-04ea377d52d3 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.852545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaf3e1-8bf0-40f3-a945-04ea377d52d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.852545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaf3e1-8bf0-40f3-a945-04ea377d52d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.853120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaf3e1-8bf0-40f3-a945-04ea377d52d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:16.854530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaf3e1-8bf0-40f3-a945-04ea377d52d3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:16.878220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 513/1018] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:18:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:18:16.908330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e96121-c9c8-479b-a2d7-7352d1ab810a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:16.957351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad3a93-cf41-4d92-85cf-b8cc76f06eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:16.958004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad3a93-cf41-4d92-85cf-b8cc76f06eea None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:16.958280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad3a93-cf41-4d92-85cf-b8cc76f06eea None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:16.958549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad3a93-cf41-4d92-85cf-b8cc76f06eea None admin] PATH_INFO: `/v3/domains/default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:16.959791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad3a93-cf41-4d92-85cf-b8cc76f06eea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:16.961297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad3a93-cf41-4d92-85cf-b8cc76f06eea None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:16.962934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 506/1019] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:16.969083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaf3e1-8bf0-40f3-a945-04ea377d52d3 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:17.081584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaf3e1-8bf0-40f3-a945-04ea377d52d3 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:17.110571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:17.120040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68425d-19da-448f-8f3d-31356e8d29aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:17.120749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68425d-19da-448f-8f3d-31356e8d29aa None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:17.121024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68425d-19da-448f-8f3d-31356e8d29aa None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:17.121293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68425d-19da-448f-8f3d-31356e8d29aa None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:17.121948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68425d-19da-448f-8f3d-31356e8d29aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:17.123383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68425d-19da-448f-8f3d-31356e8d29aa None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:17.154400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 514/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:18:16 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:18.766915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2feecf-4510-4c24-ab62-7808a6dcbc12 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:18.767726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2feecf-4510-4c24-ab62-7808a6dcbc12 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:18.767726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2feecf-4510-4c24-ab62-7808a6dcbc12 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:18.772534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 507/1021] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:18.774666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b955b7-b589-4255-842d-8a2cdb6853ec None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:18.775084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b955b7-b589-4255-842d-8a2cdb6853ec None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:18.775263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b955b7-b589-4255-842d-8a2cdb6853ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:18.794423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-72b955b7-b589-4255-842d-8a2cdb6853ec None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:18.795613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-72b955b7-b589-4255-842d-8a2cdb6853ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:18.863464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 515/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:18.875102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66e706-93de-4900-917a-561af6b71fdb None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:18.875927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66e706-93de-4900-917a-561af6b71fdb None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:18.876204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66e706-93de-4900-917a-561af6b71fdb None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:18.877811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 508/1023] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:18.882715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a577a-109a-4f99-97a6-84a26cdeead3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:18.882953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a577a-109a-4f99-97a6-84a26cdeead3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:18.883224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a577a-109a-4f99-97a6-84a26cdeead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:18.900964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-1f5a577a-109a-4f99-97a6-84a26cdeead3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:18.902139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1f5a577a-109a-4f99-97a6-84a26cdeead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:18.968776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 516/1024] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:19.092811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ee89a-3272-403c-94f7-60696423c16f None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.093125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ee89a-3272-403c-94f7-60696423c16f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.093360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ee89a-3272-403c-94f7-60696423c16f None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.095549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 509/1025] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:19.100732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1657fd-95ff-46fd-81ec-4a4ba4de02b8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.101021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1657fd-95ff-46fd-81ec-4a4ba4de02b8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.101293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1657fd-95ff-46fd-81ec-4a4ba4de02b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.120029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-de1657fd-95ff-46fd-81ec-4a4ba4de02b8 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:19.121238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-de1657fd-95ff-46fd-81ec-4a4ba4de02b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:19.131481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ee89a-3272-403c-94f7-60696423c16f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.157844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:19.169362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:19.198825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 517/1026] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:19.233005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1657fd-95ff-46fd-81ec-4a4ba4de02b8 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.245040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ee89a-3272-403c-94f7-60696423c16f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:19.279362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779c819-48b9-411a-b769-6d350a8067e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:19.279986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779c819-48b9-411a-b769-6d350a8067e1 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.280262 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779c819-48b9-411a-b769-6d350a8067e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.280560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779c819-48b9-411a-b769-6d350a8067e1 None admin] PATH_INFO: `/v3/services/image` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.281145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a779c819-48b9-411a-b769-6d350a8067e1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:19.283217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a779c819-48b9-411a-b769-6d350a8067e1 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-a779c819-48b9-411a-b769-6d350a8067e1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:19.291984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:19.293001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:19.293861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 510/1027] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:19.299840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779c819-48b9-411a-b769-6d350a8067e1 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.353240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1657fd-95ff-46fd-81ec-4a4ba4de02b8 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:19.388995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd77a4-0f7a-4fb9-a56d-81c7c24feeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:19.389633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd77a4-0f7a-4fb9-a56d-81c7c24feeb1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.389895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd77a4-0f7a-4fb9-a56d-81c7c24feeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.390160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd77a4-0f7a-4fb9-a56d-81c7c24feeb1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.401213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd77a4-0f7a-4fb9-a56d-81c7c24feeb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:19.402388 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-68fd77a4-0f7a-4fb9-a56d-81c7c24feeb1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:19.406350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:19.412617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:19.413460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:19.414093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779c819-48b9-411a-b769-6d350a8067e1 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:19.414177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 518/1028] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:19.414177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd77a4-0f7a-4fb9-a56d-81c7c24feeb1 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.443839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a383b01-f066-4ec4-9ab9-9ff4a0d6d83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:19.444441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a383b01-f066-4ec4-9ab9-9ff4a0d6d83a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.444768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a383b01-f066-4ec4-9ab9-9ff4a0d6d83a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.445274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a383b01-f066-4ec4-9ab9-9ff4a0d6d83a None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.445659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a383b01-f066-4ec4-9ab9-9ff4a0d6d83a None admin] RBAC: Adding query filter params (name=image) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:19.445958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a383b01-f066-4ec4-9ab9-9ff4a0d6d83a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:19.447323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a383b01-f066-4ec4-9ab9-9ff4a0d6d83a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:19.447964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:19.456543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 511/1029] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:19.458634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a383b01-f066-4ec4-9ab9-9ff4a0d6d83a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.522749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd77a4-0f7a-4fb9-a56d-81c7c24feeb1 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:19.556960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803446-8ece-45ca-8092-2384e8fdddbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:19.557559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803446-8ece-45ca-8092-2384e8fdddbe None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.557839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803446-8ece-45ca-8092-2384e8fdddbe None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.558030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a383b01-f066-4ec4-9ab9-9ff4a0d6d83a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:19.558135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803446-8ece-45ca-8092-2384e8fdddbe None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.558743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803446-8ece-45ca-8092-2384e8fdddbe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:19.559037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803446-8ece-45ca-8092-2384e8fdddbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:19.560405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803446-8ece-45ca-8092-2384e8fdddbe None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:19.573351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 519/1030] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:19.573351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803446-8ece-45ca-8092-2384e8fdddbe None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.595505 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b1b38-a8e2-4582-8979-907634c62d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:19.596112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b1b38-a8e2-4582-8979-907634c62d82 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.596389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b1b38-a8e2-4582-8979-907634c62d82 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.596693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b1b38-a8e2-4582-8979-907634c62d82 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.597619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b1b38-a8e2-4582-8979-907634c62d82 None admin] RBAC: Adding query filter params (type=image) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:19.597948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b1b38-a8e2-4582-8979-907634c62d82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:19.599551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b1b38-a8e2-4582-8979-907634c62d82 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:19.604344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 512/1031] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:19.605162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b1b38-a8e2-4582-8979-907634c62d82 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.633247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:19.668238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:19.678221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803446-8ece-45ca-8092-2384e8fdddbe None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:19.706227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b1b38-a8e2-4582-8979-907634c62d82 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:19.710858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9f83e-f16a-4677-b922-03e87b64a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:19.711486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9f83e-f16a-4677-b922-03e87b64a63f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.711763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9f83e-f16a-4677-b922-03e87b64a63f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.712032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9f83e-f16a-4677-b922-03e87b64a63f None admin] PATH_INFO: `/v3/domains/default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.713150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9f83e-f16a-4677-b922-03e87b64a63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:19.714586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9f83e-f16a-4677-b922-03e87b64a63f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:19.716145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 520/1032] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:19.716924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9f83e-f16a-4677-b922-03e87b64a63f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.742440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488ffb4-2afa-4a1b-92fa-5c06943a82d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:19.743062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488ffb4-2afa-4a1b-92fa-5c06943a82d6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.743335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488ffb4-2afa-4a1b-92fa-5c06943a82d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.743601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488ffb4-2afa-4a1b-92fa-5c06943a82d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.754600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488ffb4-2afa-4a1b-92fa-5c06943a82d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:19.756040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488ffb4-2afa-4a1b-92fa-5c06943a82d6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-9488ffb4-2afa-4a1b-92fa-5c06943a82d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:19.771027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:19.772632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:19.773745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:19.773745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:19.773745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:19.773745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 513/1033] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:19.773745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488ffb4-2afa-4a1b-92fa-5c06943a82d6 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.827681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9f83e-f16a-4677-b922-03e87b64a63f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:19.851013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:19.862299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046238d2-3e8b-45bd-a0d2-5d590f06d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:19.862922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046238d2-3e8b-45bd-a0d2-5d590f06d843 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.863196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046238d2-3e8b-45bd-a0d2-5d590f06d843 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.863463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046238d2-3e8b-45bd-a0d2-5d590f06d843 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.864154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046238d2-3e8b-45bd-a0d2-5d590f06d843 None admin] RBAC: Adding query filter params (interface=public, service_id=6a5c850ce3d3477ab5acd3da912aa43b, region_id=RegionOne) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:19.864458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046238d2-3e8b-45bd-a0d2-5d590f06d843 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:19.865847 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046238d2-3e8b-45bd-a0d2-5d590f06d843 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:19.871291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 521/1034] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/endpoints?service_id=6a5c850ce3d3477ab5acd3da912aa43b&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:19.872060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046238d2-3e8b-45bd-a0d2-5d590f06d843 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.872865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488ffb4-2afa-4a1b-92fa-5c06943a82d6 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:19.896724 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:19.909052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34cee3-cfc9-45b7-93b8-c2fe32193607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:19.909658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34cee3-cfc9-45b7-93b8-c2fe32193607 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:19.909943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34cee3-cfc9-45b7-93b8-c2fe32193607 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:19.910214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34cee3-cfc9-45b7-93b8-c2fe32193607 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:19.910853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34cee3-cfc9-45b7-93b8-c2fe32193607 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:19.912275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e34cee3-cfc9-45b7-93b8-c2fe32193607 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:19.948552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 514/1035] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:18:19 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:19.948552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34cee3-cfc9-45b7-93b8-c2fe32193607 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:19.980794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046238d2-3e8b-45bd-a0d2-5d590f06d843 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:20.021220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a4419-07f5-4140-941f-191f6bb694db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:20.022341 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a4419-07f5-4140-941f-191f6bb694db None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:20.022734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a4419-07f5-4140-941f-191f6bb694db None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:20.023011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a4419-07f5-4140-941f-191f6bb694db None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:20.023665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a4419-07f5-4140-941f-191f6bb694db None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:20.023963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a4419-07f5-4140-941f-191f6bb694db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:20.025614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a4419-07f5-4140-941f-191f6bb694db None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:20.044305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 522/1036] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:20.045331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a4419-07f5-4140-941f-191f6bb694db None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:20.067905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34cee3-cfc9-45b7-93b8-c2fe32193607 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:20.110086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff81b9-6b16-45d5-b3ee-cfd84695167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:20.110705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff81b9-6b16-45d5-b3ee-cfd84695167f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:20.110984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff81b9-6b16-45d5-b3ee-cfd84695167f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:20.111267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff81b9-6b16-45d5-b3ee-cfd84695167f None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:20.111886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff81b9-6b16-45d5-b3ee-cfd84695167f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:20.113348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff81b9-6b16-45d5-b3ee-cfd84695167f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:20.114405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:20.121941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 515/1037] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 00:18:19 2022] GET /identity/v3/services => generated 1957 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:18:20.148213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:20.168542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a4419-07f5-4140-941f-191f6bb694db None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:20.207760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d6429-0fe6-417d-97fa-7b1a6e2a8a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:20.208407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d6429-0fe6-417d-97fa-7b1a6e2a8a5f None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:20.208720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d6429-0fe6-417d-97fa-7b1a6e2a8a5f None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:20.208985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d6429-0fe6-417d-97fa-7b1a6e2a8a5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:20.217113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d6429-0fe6-417d-97fa-7b1a6e2a8a5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:20.218586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d6429-0fe6-417d-97fa-7b1a6e2a8a5f None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-0a6d6429-0fe6-417d-97fa-7b1a6e2a8a5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:18:20.233085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:20.234212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:20.235420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 523/1038] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 00:18:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:20.242348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff81b9-6b16-45d5-b3ee-cfd84695167f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:20.353602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff81b9-6b16-45d5-b3ee-cfd84695167f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:20.387601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f2528-5746-4843-a106-1dd7fa535ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:20.388130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f2528-5746-4843-a106-1dd7fa535ac7 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:20.388406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f2528-5746-4843-a106-1dd7fa535ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:20.388692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f2528-5746-4843-a106-1dd7fa535ac7 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:20.389423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f2528-5746-4843-a106-1dd7fa535ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:20.389728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f2528-5746-4843-a106-1dd7fa535ac7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:20.391447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f2528-5746-4843-a106-1dd7fa535ac7 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:20.414782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 516/1039] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 00:18:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:20.416431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d6429-0fe6-417d-97fa-7b1a6e2a8a5f None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:20.518683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d6429-0fe6-417d-97fa-7b1a6e2a8a5f None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:20.551889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3c3f-cfba-4768-87e3-01b394bb9b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:20.552669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3c3f-cfba-4768-87e3-01b394bb9b2e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:20.553044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3c3f-cfba-4768-87e3-01b394bb9b2e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:20.553414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3c3f-cfba-4768-87e3-01b394bb9b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:20.568642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:20.627327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db3c3f-cfba-4768-87e3-01b394bb9b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:20.628803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db3c3f-cfba-4768-87e3-01b394bb9b2e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:20.733345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 524/1040] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 00:18:20.734953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f2528-5746-4843-a106-1dd7fa535ac7 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:20.794733 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:20.839458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f2528-5746-4843-a106-1dd7fa535ac7 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:20.873589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cb3fb-d663-4e86-919b-25c6545263ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:20.874213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cb3fb-d663-4e86-919b-25c6545263ec None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:20.874493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cb3fb-d663-4e86-919b-25c6545263ec None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:20.874765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cb3fb-d663-4e86-919b-25c6545263ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:20.885013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cb3fb-d663-4e86-919b-25c6545263ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:20.886459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cb3fb-d663-4e86-919b-25c6545263ec None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-753cb3fb-d663-4e86-919b-25c6545263ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:20.902064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:20.903012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:20.903917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:20.903917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:20.903917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:20.908348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 517/1041] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:20.910114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3c3f-cfba-4768-87e3-01b394bb9b2e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:20.957083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:21.014709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3c3f-cfba-4768-87e3-01b394bb9b2e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:21.050391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078cf49-d9a6-48b6-b6b6-7a8e31d291fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:21.051135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078cf49-d9a6-48b6-b6b6-7a8e31d291fc None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:21.051511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078cf49-d9a6-48b6-b6b6-7a8e31d291fc None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:21.051917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078cf49-d9a6-48b6-b6b6-7a8e31d291fc None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:21.052688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078cf49-d9a6-48b6-b6b6-7a8e31d291fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:21.053113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078cf49-d9a6-48b6-b6b6-7a8e31d291fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:21.054585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078cf49-d9a6-48b6-b6b6-7a8e31d291fc None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:21.071205 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 525/1042] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:21.072976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cb3fb-d663-4e86-919b-25c6545263ec None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:21.178203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cb3fb-d663-4e86-919b-25c6545263ec None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:21.187280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:21.213851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158becf8-8c0d-4d5d-8571-e90c2aee1091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:21.214665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158becf8-8c0d-4d5d-8571-e90c2aee1091 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:21.215044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158becf8-8c0d-4d5d-8571-e90c2aee1091 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:21.215414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158becf8-8c0d-4d5d-8571-e90c2aee1091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:21.223808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158becf8-8c0d-4d5d-8571-e90c2aee1091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:21.225713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158becf8-8c0d-4d5d-8571-e90c2aee1091 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-158becf8-8c0d-4d5d-8571-e90c2aee1091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:18:21.242463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:21.243369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:21.244279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:18:21.244279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:18:21.244279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:18:21.244279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:18:21.244279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:18:21.244279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:18:21.244279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:21.244279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:21.245367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 518/1043] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:21.249651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078cf49-d9a6-48b6-b6b6-7a8e31d291fc None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:21.356834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078cf49-d9a6-48b6-b6b6-7a8e31d291fc None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:21.359148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:21.392352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a08a0-328f-423d-ba6b-58039a9fc791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:21.392985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a08a0-328f-423d-ba6b-58039a9fc791 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:21.393258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a08a0-328f-423d-ba6b-58039a9fc791 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:21.393521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a08a0-328f-423d-ba6b-58039a9fc791 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:21.394262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a08a0-328f-423d-ba6b-58039a9fc791 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:21.394559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a08a0-328f-423d-ba6b-58039a9fc791 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:21.396029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a08a0-328f-423d-ba6b-58039a9fc791 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:21.412680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 526/1044] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:21.414893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158becf8-8c0d-4d5d-8571-e90c2aee1091 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:21.519452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158becf8-8c0d-4d5d-8571-e90c2aee1091 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:21.553248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59550e9d-8b25-4f27-ad4c-90d44d68e835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:21.553855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59550e9d-8b25-4f27-ad4c-90d44d68e835 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:21.554140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59550e9d-8b25-4f27-ad4c-90d44d68e835 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:21.554411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59550e9d-8b25-4f27-ad4c-90d44d68e835 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:21.555157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59550e9d-8b25-4f27-ad4c-90d44d68e835 None admin] RBAC: Adding query filter params (role.id=448c7e73924b47a1b6af2135604b5dbb, scope.project.id=ed93469862874c75951d95e28929c904, user.id=fb204040ab2f4168b35c3e78bacb849f) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:21.555467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59550e9d-8b25-4f27-ad4c-90d44d68e835 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:21.556935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59550e9d-8b25-4f27-ad4c-90d44d68e835 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:21.567987 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 519/1045] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:21 2022] GET /identity/v3/role_assignments?user.id=fb204040ab2f4168b35c3e78bacb849f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=448c7e73924b47a1b6af2135604b5dbb => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:21.975795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8552f-4a3b-44e4-9c62-0e475f3349c9 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:21.976345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8552f-4a3b-44e4-9c62-0e475f3349c9 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:21.976345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8552f-4a3b-44e4-9c62-0e475f3349c9 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:21.977732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 527/1046] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:21.982919 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818a6a0-a7ce-4bc5-9f21-6b3046ee7718 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:21.983304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818a6a0-a7ce-4bc5-9f21-6b3046ee7718 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:21.983481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818a6a0-a7ce-4bc5-9f21-6b3046ee7718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:22.003555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1818a6a0-a7ce-4bc5-9f21-6b3046ee7718 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:22.004765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1818a6a0-a7ce-4bc5-9f21-6b3046ee7718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:22.031207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:22.072848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c6faf-188a-40b2-b9ab-ad4b6d68e6e1 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:22.073175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c6faf-188a-40b2-b9ab-ad4b6d68e6e1 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:22.073416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c6faf-188a-40b2-b9ab-ad4b6d68e6e1 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:22.074611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 520/1047] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:22.074957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 528/1048] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:22.080700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63390d0-37d6-4938-badd-af97e0350abe None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:22.080989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63390d0-37d6-4938-badd-af97e0350abe None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:22.081259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63390d0-37d6-4938-badd-af97e0350abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:22.099404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b63390d0-37d6-4938-badd-af97e0350abe None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:22.100603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b63390d0-37d6-4938-badd-af97e0350abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:22.167904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 521/1049] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:22.347702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c6faf-188a-40b2-b9ab-ad4b6d68e6e1 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:22.447247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63390d0-37d6-4938-badd-af97e0350abe None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:22.449748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c6faf-188a-40b2-b9ab-ad4b6d68e6e1 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:22.487122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e9d91-35e2-4a35-926f-17ea07df1ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:22.487672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9d91-35e2-4a35-926f-17ea07df1ccc None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:22.487948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9d91-35e2-4a35-926f-17ea07df1ccc None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:22.488223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e9d91-35e2-4a35-926f-17ea07df1ccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:22.498531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e9d91-35e2-4a35-926f-17ea07df1ccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-781e9d91-35e2-4a35-926f-17ea07df1ccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:22.503463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:22.504440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:22.505946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 529/1050] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:18:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:22.510421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e9d91-35e2-4a35-926f-17ea07df1ccc None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:22.539344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:22.547383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:22.557063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63390d0-37d6-4938-badd-af97e0350abe None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:22.596345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f622961-fc95-4b9b-8cc7-671e999762b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:22.597025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f622961-fc95-4b9b-8cc7-671e999762b6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:22.597300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f622961-fc95-4b9b-8cc7-671e999762b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:22.597564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f622961-fc95-4b9b-8cc7-671e999762b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:22.598155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f622961-fc95-4b9b-8cc7-671e999762b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:22.599797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f622961-fc95-4b9b-8cc7-671e999762b6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-7f622961-fc95-4b9b-8cc7-671e999762b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:22.608822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:22.609787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:22.610614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 522/1051] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 1 00:18:22 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:22.616071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f622961-fc95-4b9b-8cc7-671e999762b6 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:22.625006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e9d91-35e2-4a35-926f-17ea07df1ccc None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:22.664291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dfb22-1fd3-4a36-a17c-43b5618dafdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:22.664976 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dfb22-1fd3-4a36-a17c-43b5618dafdd None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:22.665367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dfb22-1fd3-4a36-a17c-43b5618dafdd None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:22.665638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dfb22-1fd3-4a36-a17c-43b5618dafdd None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:22.666456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dfb22-1fd3-4a36-a17c-43b5618dafdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:22.666754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dfb22-1fd3-4a36-a17c-43b5618dafdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:22.668114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dfb22-1fd3-4a36-a17c-43b5618dafdd None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:22.681828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 530/1052] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:18:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:22.682642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dfb22-1fd3-4a36-a17c-43b5618dafdd None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:22.721332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f622961-fc95-4b9b-8cc7-671e999762b6 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:22.759001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44b6be-4e5e-4e45-b0fb-859d4cdc1db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:22.759536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44b6be-4e5e-4e45-b0fb-859d4cdc1db7 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:22.759814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44b6be-4e5e-4e45-b0fb-859d4cdc1db7 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:22.760090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44b6be-4e5e-4e45-b0fb-859d4cdc1db7 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:22.760752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44b6be-4e5e-4e45-b0fb-859d4cdc1db7 None admin] RBAC: Adding query filter params (name=image) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:22.761059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44b6be-4e5e-4e45-b0fb-859d4cdc1db7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:22.762462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44b6be-4e5e-4e45-b0fb-859d4cdc1db7 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:22.768386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 523/1053] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:22.773985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44b6be-4e5e-4e45-b0fb-859d4cdc1db7 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:22.784937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:22.798440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:22.803189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dfb22-1fd3-4a36-a17c-43b5618dafdd None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:22.840423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f764a4-e4e4-4e50-ae3f-982c262d8d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:22.841170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f764a4-e4e4-4e50-ae3f-982c262d8d46 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:22.841667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f764a4-e4e4-4e50-ae3f-982c262d8d46 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:22.841779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f764a4-e4e4-4e50-ae3f-982c262d8d46 None admin] PATH_INFO: `/v3/users/2124c4c0223a4de18cdafb58d19473f8` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:22.856253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f764a4-e4e4-4e50-ae3f-982c262d8d46 None admin] RBAC: Authorizing `identity:get_user(user_id=2124c4c0223a4de18cdafb58d19473f8)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:22.857720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f764a4-e4e4-4e50-ae3f-982c262d8d46 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:22.859553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 531/1054] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:22 2022] GET /identity/v3/users/2124c4c0223a4de18cdafb58d19473f8 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:22.865027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f764a4-e4e4-4e50-ae3f-982c262d8d46 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:22.885902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44b6be-4e5e-4e45-b0fb-859d4cdc1db7 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:22.923015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3d579-7f7a-4132-8ce3-b5a429a5ea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:22.923679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3d579-7f7a-4132-8ce3-b5a429a5ea62 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:22.923985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3d579-7f7a-4132-8ce3-b5a429a5ea62 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:22.924271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3d579-7f7a-4132-8ce3-b5a429a5ea62 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:22.924889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3d579-7f7a-4132-8ce3-b5a429a5ea62 None admin] RBAC: Adding query filter params (type=image) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:22.925191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3d579-7f7a-4132-8ce3-b5a429a5ea62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:22.926557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba3d579-7f7a-4132-8ce3-b5a429a5ea62 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:22.936827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 524/1055] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:22.938426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3d579-7f7a-4132-8ce3-b5a429a5ea62 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:22.979965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f764a4-e4e4-4e50-ae3f-982c262d8d46 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:23.017590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a294ee-f2fa-4c31-99d4-7643823b8360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:23.018457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a294ee-f2fa-4c31-99d4-7643823b8360 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:23.018732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a294ee-f2fa-4c31-99d4-7643823b8360 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:23.018996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a294ee-f2fa-4c31-99d4-7643823b8360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:23.025047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:23.035356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:23.043041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3d579-7f7a-4132-8ce3-b5a429a5ea62 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:23.083085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c59df-82d1-4470-8138-35ae0aeb2b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:23.083694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c59df-82d1-4470-8138-35ae0aeb2b12 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:23.083969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c59df-82d1-4470-8138-35ae0aeb2b12 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:23.084237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c59df-82d1-4470-8138-35ae0aeb2b12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:23.084939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c59df-82d1-4470-8138-35ae0aeb2b12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:23.086300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c59df-82d1-4470-8138-35ae0aeb2b12 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:23.096322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a294ee-f2fa-4c31-99d4-7643823b8360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:23.098343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a294ee-f2fa-4c31-99d4-7643823b8360 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:23.109581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 525/1056] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:18:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:23.208206 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 532/1057] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:22 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 00:18:23.214230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c59df-82d1-4470-8138-35ae0aeb2b12 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:23.343665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c59df-82d1-4470-8138-35ae0aeb2b12 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:23.379233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0487e-2ae5-48e3-a082-44ac2c701736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:23.380043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0487e-2ae5-48e3-a082-44ac2c701736 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:23.380418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0487e-2ae5-48e3-a082-44ac2c701736 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:23.380810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0487e-2ae5-48e3-a082-44ac2c701736 None admin] PATH_INFO: `/v3/projects/d3d55145d5224f3d886c3d4c644de8f0` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:23.382026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0487e-2ae5-48e3-a082-44ac2c701736 None admin] RBAC: Authorizing `identity:get_project(project_id=d3d55145d5224f3d886c3d4c644de8f0)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:23.383557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0487e-2ae5-48e3-a082-44ac2c701736 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:23.385609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 526/1058] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:18:23 2022] GET /identity/v3/projects/d3d55145d5224f3d886c3d4c644de8f0 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:23.392398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a294ee-f2fa-4c31-99d4-7643823b8360 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:23.439233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:23.483280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d00d7-5cae-421b-b01b-42a38e001bb5 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:23.483622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d00d7-5cae-421b-b01b-42a38e001bb5 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:23.483826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d00d7-5cae-421b-b01b-42a38e001bb5 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:23.485649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 527/1059] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:23.490586 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98485f57-c030-436a-9ee7-0401dd0d1c11 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:23.490865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98485f57-c030-436a-9ee7-0401dd0d1c11 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:23.491141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98485f57-c030-436a-9ee7-0401dd0d1c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:23.498028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a294ee-f2fa-4c31-99d4-7643823b8360 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:23.502616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:23.514608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-98485f57-c030-436a-9ee7-0401dd0d1c11 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:23.516333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-98485f57-c030-436a-9ee7-0401dd0d1c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:23.535896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd5a4f-9914-421f-9cd4-6079f4478ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:23.536515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd5a4f-9914-421f-9cd4-6079f4478ad4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:23.536802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd5a4f-9914-421f-9cd4-6079f4478ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:23.537071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd5a4f-9914-421f-9cd4-6079f4478ad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:23.537788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd5a4f-9914-421f-9cd4-6079f4478ad4 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=d3d55145d5224f3d886c3d4c644de8f0, user.id=2124c4c0223a4de18cdafb58d19473f8) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:23.538315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd5a4f-9914-421f-9cd4-6079f4478ad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:23.539511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd5a4f-9914-421f-9cd4-6079f4478ad4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:23.545243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 533/1060] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:23 2022] GET /identity/v3/role_assignments?user.id=2124c4c0223a4de18cdafb58d19473f8&scope.project.id=d3d55145d5224f3d886c3d4c644de8f0&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:23.592599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 528/1061] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:23.875947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd5a4f-9914-421f-9cd4-6079f4478ad4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:23.980431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd5a4f-9914-421f-9cd4-6079f4478ad4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:23.992907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:24.019568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043a641-9e84-46ca-a877-2437fb4ccc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:24.020201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043a641-9e84-46ca-a877-2437fb4ccc7d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:24.020477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043a641-9e84-46ca-a877-2437fb4ccc7d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:24.020778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043a641-9e84-46ca-a877-2437fb4ccc7d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:24.031323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043a641-9e84-46ca-a877-2437fb4ccc7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-0043a641-9e84-46ca-a877-2437fb4ccc7d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:24.036445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:24.037458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:24.038280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:24.040809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 534/1062] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 00:18:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:24.043668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98485f57-c030-436a-9ee7-0401dd0d1c11 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:24.146077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98485f57-c030-436a-9ee7-0401dd0d1c11 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:24.180864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb1dcc-66a2-46cf-9798-2fc2762726f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:24.181476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb1dcc-66a2-46cf-9798-2fc2762726f6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:24.181751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb1dcc-66a2-46cf-9798-2fc2762726f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:24.182020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb1dcc-66a2-46cf-9798-2fc2762726f6 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:24.182679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb1dcc-66a2-46cf-9798-2fc2762726f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:24.182977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb1dcc-66a2-46cf-9798-2fc2762726f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:24.184346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb1dcc-66a2-46cf-9798-2fc2762726f6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:24.185868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:24.196647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 529/1063] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 00:18:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:24.200105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043a641-9e84-46ca-a877-2437fb4ccc7d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:24.314110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043a641-9e84-46ca-a877-2437fb4ccc7d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:24.348031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb3d2-ec1c-4008-9668-82fb4f455316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:24.348900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb3d2-ec1c-4008-9668-82fb4f455316 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:24.349632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb3d2-ec1c-4008-9668-82fb4f455316 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:24.350091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb3d2-ec1c-4008-9668-82fb4f455316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:24.360792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bb3d2-ec1c-4008-9668-82fb4f455316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:24.362408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bb3d2-ec1c-4008-9668-82fb4f455316 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-7e5bb3d2-ec1c-4008-9668-82fb4f455316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:24.377325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:24.378288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:24.379202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:24.379202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:24.379202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:24.385585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 535/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:24.386575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb1dcc-66a2-46cf-9798-2fc2762726f6 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:24.490628 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb1dcc-66a2-46cf-9798-2fc2762726f6 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:24.526568 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730c0c3-6e97-49fd-8390-055779fca307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:24.527467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730c0c3-6e97-49fd-8390-055779fca307 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:24.527927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730c0c3-6e97-49fd-8390-055779fca307 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:24.528373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730c0c3-6e97-49fd-8390-055779fca307 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:24.529293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730c0c3-6e97-49fd-8390-055779fca307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:24.529607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730c0c3-6e97-49fd-8390-055779fca307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:24.531007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730c0c3-6e97-49fd-8390-055779fca307 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:24.548319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 530/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:24.549330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb3d2-ec1c-4008-9668-82fb4f455316 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:24.573118 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:24.657654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb3d2-ec1c-4008-9668-82fb4f455316 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:24.695936 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f7fa7-bca8-460d-bb9f-94fc750218f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:24.696602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f7fa7-bca8-460d-bb9f-94fc750218f2 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:24.699325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f7fa7-bca8-460d-bb9f-94fc750218f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:24.699325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f7fa7-bca8-460d-bb9f-94fc750218f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:24.708470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f7fa7-bca8-460d-bb9f-94fc750218f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:24.708470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f7fa7-bca8-460d-bb9f-94fc750218f2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-3f3f7fa7-bca8-460d-bb9f-94fc750218f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:18:24.718390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:24.719958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:18:24.720909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:24.725444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 536/1066] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 00:18:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:24.726481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730c0c3-6e97-49fd-8390-055779fca307 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:24.791779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:24.840352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730c0c3-6e97-49fd-8390-055779fca307 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:24.877971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e5bcf-4c33-49eb-9aa0-363efd153f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:24.878935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e5bcf-4c33-49eb-9aa0-363efd153f52 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:24.879410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e5bcf-4c33-49eb-9aa0-363efd153f52 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:24.879903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e5bcf-4c33-49eb-9aa0-363efd153f52 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:24.880882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e5bcf-4c33-49eb-9aa0-363efd153f52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:24.881418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e5bcf-4c33-49eb-9aa0-363efd153f52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:24.883043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e5bcf-4c33-49eb-9aa0-363efd153f52 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:24.908026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f7fa7-bca8-460d-bb9f-94fc750218f2 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:24.917862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 531/1067] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 00:18:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:24.992866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:25.012701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f7fa7-bca8-460d-bb9f-94fc750218f2 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:25.049304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d72517-13e9-4bdd-ac59-af9910964d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:25.050010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d72517-13e9-4bdd-ac59-af9910964d7b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.050386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d72517-13e9-4bdd-ac59-af9910964d7b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.050754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d72517-13e9-4bdd-ac59-af9910964d7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.062103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d72517-13e9-4bdd-ac59-af9910964d7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:25.063852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d72517-13e9-4bdd-ac59-af9910964d7b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:25.069154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099e4ef-c379-4fb1-a7ac-b211833717d2 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.069644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099e4ef-c379-4fb1-a7ac-b211833717d2 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.069893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099e4ef-c379-4fb1-a7ac-b211833717d2 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.071241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 532/1068] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:25.076209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70186d5d-ea2d-42d6-a39a-39ba6123df0f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.076512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70186d5d-ea2d-42d6-a39a-39ba6123df0f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.076783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70186d5d-ea2d-42d6-a39a-39ba6123df0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f6d72517-13e9-4bdd-ac59-af9910964d7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:25.090861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:25.091869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:25.093021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:25.093021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:25.093021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:25.094932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 537/1069] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:25.097853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-70186d5d-ea2d-42d6-a39a-39ba6123df0f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:25.099284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-70186d5d-ea2d-42d6-a39a-39ba6123df0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:25.101279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d72517-13e9-4bdd-ac59-af9910964d7b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:25.172855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 533/1070] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:25.209880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d72517-13e9-4bdd-ac59-af9910964d7b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:25.242860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:25.251942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da86ead-99d0-4999-b02e-dcab3b7d8c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:25.251942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da86ead-99d0-4999-b02e-dcab3b7d8c31 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.251942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da86ead-99d0-4999-b02e-dcab3b7d8c31 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.251942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da86ead-99d0-4999-b02e-dcab3b7d8c31 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.252532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da86ead-99d0-4999-b02e-dcab3b7d8c31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:25.252919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da86ead-99d0-4999-b02e-dcab3b7d8c31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:25.254580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da86ead-99d0-4999-b02e-dcab3b7d8c31 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:25.272131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 538/1071] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:25.273404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70186d5d-ea2d-42d6-a39a-39ba6123df0f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:25.321106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:25.378188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70186d5d-ea2d-42d6-a39a-39ba6123df0f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:25.412030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bab8d7-b2e1-4174-aca8-3ff8d9a2d663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:25.412685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bab8d7-b2e1-4174-aca8-3ff8d9a2d663 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.412964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bab8d7-b2e1-4174-aca8-3ff8d9a2d663 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.413233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bab8d7-b2e1-4174-aca8-3ff8d9a2d663 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.419901 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bab8d7-b2e1-4174-aca8-3ff8d9a2d663 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:25.421364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bab8d7-b2e1-4174-aca8-3ff8d9a2d663 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-91bab8d7-b2e1-4174-aca8-3ff8d9a2d663 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:18:25.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:25.434254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:25.435774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:18:25.435774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:18:25.435774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:18:25.435774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:18:25.435774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:18:25.435774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:18:25.435774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:25.435774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:25.435774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 534/1072] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:25.440688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da86ead-99d0-4999-b02e-dcab3b7d8c31 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:25.448908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bab8d7-b2e1-4174-aca8-3ff8d9a2d663 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:25.540100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da86ead-99d0-4999-b02e-dcab3b7d8c31 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:25.549810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bab8d7-b2e1-4174-aca8-3ff8d9a2d663 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:25.552423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:25.577512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7117d-a036-4213-9c4c-eb2872fed812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:25.578073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7117d-a036-4213-9c4c-eb2872fed812 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.578350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7117d-a036-4213-9c4c-eb2872fed812 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.578765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7117d-a036-4213-9c4c-eb2872fed812 None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.579465 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7117d-a036-4213-9c4c-eb2872fed812 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:25.579773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7117d-a036-4213-9c4c-eb2872fed812 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:25.581241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7117d-a036-4213-9c4c-eb2872fed812 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:25.585503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7bec0-e5de-4631-b44c-f40ffede5309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:25.586099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7bec0-e5de-4631-b44c-f40ffede5309 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.586373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7bec0-e5de-4631-b44c-f40ffede5309 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.586639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7bec0-e5de-4631-b44c-f40ffede5309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.592730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 539/1073] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:25.594802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca26e5-4903-4c25-82a9-4b51a33e3dac None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.595090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca26e5-4903-4c25-82a9-4b51a33e3dac None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.595368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca26e5-4903-4c25-82a9-4b51a33e3dac None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.596599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 540/1074] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:25.599451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca26e5-4903-4c25-82a9-4b51a33e3dac None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:25.651913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:25.661866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7bec0-e5de-4631-b44c-f40ffede5309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:25.663326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7bec0-e5de-4631-b44c-f40ffede5309 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:25.704173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca26e5-4903-4c25-82a9-4b51a33e3dac None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:25.739912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62647ab-7d7b-42da-9ad3-753168b59bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:25.740596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62647ab-7d7b-42da-9ad3-753168b59bc2 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.740880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62647ab-7d7b-42da-9ad3-753168b59bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.741145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62647ab-7d7b-42da-9ad3-753168b59bc2 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/fb204040ab2f4168b35c3e78bacb849f/roles/448c7e73924b47a1b6af2135604b5dbb` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.743302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62647ab-7d7b-42da-9ad3-753168b59bc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=fb204040ab2f4168b35c3e78bacb849f, role_id=448c7e73924b47a1b6af2135604b5dbb)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:25.744759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62647ab-7d7b-42da-9ad3-753168b59bc2 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:25.757536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 541/1075] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:18:25 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/fb204040ab2f4168b35c3e78bacb849f/roles/448c7e73924b47a1b6af2135604b5dbb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:18:25.758702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d254e0d-ceec-47a9-857e-92d7510c556b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.758983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d254e0d-ceec-47a9-857e-92d7510c556b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.759251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d254e0d-ceec-47a9-857e-92d7510c556b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.766469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 535/1076] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 00:18:25.772320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7bec0-e5de-4631-b44c-f40ffede5309 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:25.776596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:25.785570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-9d254e0d-ceec-47a9-857e-92d7510c556b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:25.786796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9d254e0d-ceec-47a9-857e-92d7510c556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:25.864636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 542/1077] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:25.907053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7bec0-e5de-4631-b44c-f40ffede5309 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:25.945348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d568b6-e365-49d5-8fc1-2561876a12b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:25.945974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d568b6-e365-49d5-8fc1-2561876a12b2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:25.946252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d568b6-e365-49d5-8fc1-2561876a12b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:25.946522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d568b6-e365-49d5-8fc1-2561876a12b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:25.957736 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d568b6-e365-49d5-8fc1-2561876a12b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:25.959196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d568b6-e365-49d5-8fc1-2561876a12b2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-78d568b6-e365-49d5-8fc1-2561876a12b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:25.974221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:25.975212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:25.976659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:25.976659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:25.976659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:25.976659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 536/1078] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:25.981178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d254e0d-ceec-47a9-857e-92d7510c556b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:26.024476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:26.084403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d254e0d-ceec-47a9-857e-92d7510c556b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:26.119113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aad6ca-a14a-4cb8-b4c4-6dde2fd72963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:26.119631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aad6ca-a14a-4cb8-b4c4-6dde2fd72963 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:26.119904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aad6ca-a14a-4cb8-b4c4-6dde2fd72963 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:26.120169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aad6ca-a14a-4cb8-b4c4-6dde2fd72963 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:26.120847 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aad6ca-a14a-4cb8-b4c4-6dde2fd72963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:26.121143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aad6ca-a14a-4cb8-b4c4-6dde2fd72963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:26.122502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aad6ca-a14a-4cb8-b4c4-6dde2fd72963 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:26.133148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d568b6-e365-49d5-8fc1-2561876a12b2 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:26.134108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 543/1079] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:26.198916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:26.257592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d568b6-e365-49d5-8fc1-2561876a12b2 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:26.295346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62312e49-6d14-422f-8429-abd2e28f3113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:26.295975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62312e49-6d14-422f-8429-abd2e28f3113 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:26.296248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62312e49-6d14-422f-8429-abd2e28f3113 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:26.296528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62312e49-6d14-422f-8429-abd2e28f3113 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:26.307589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62312e49-6d14-422f-8429-abd2e28f3113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-62312e49-6d14-422f-8429-abd2e28f3113 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:26.312342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:26.313359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:26.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 537/1080] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:18:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:26.319598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aad6ca-a14a-4cb8-b4c4-6dde2fd72963 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:26.426771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aad6ca-a14a-4cb8-b4c4-6dde2fd72963 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:26.428245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:26.461039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e146dfb-bb09-440b-9203-bea324f5c04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:26.461634 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e146dfb-bb09-440b-9203-bea324f5c04b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:26.461906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e146dfb-bb09-440b-9203-bea324f5c04b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:26.462171 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e146dfb-bb09-440b-9203-bea324f5c04b None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:26.462815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e146dfb-bb09-440b-9203-bea324f5c04b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:26.463108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e146dfb-bb09-440b-9203-bea324f5c04b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:26.464460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e146dfb-bb09-440b-9203-bea324f5c04b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:26.472870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 544/1081] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:18:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:26.479047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62312e49-6d14-422f-8429-abd2e28f3113 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:26.575105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62312e49-6d14-422f-8429-abd2e28f3113 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:26.590305 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:26.608411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe868bdd-5791-4e08-88cf-26cb9bc884ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:26.609059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe868bdd-5791-4e08-88cf-26cb9bc884ee None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:26.609934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe868bdd-5791-4e08-88cf-26cb9bc884ee None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:26.610161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe868bdd-5791-4e08-88cf-26cb9bc884ee None admin] PATH_INFO: `/v3/users/2124c4c0223a4de18cdafb58d19473f8` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:26.611277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe868bdd-5791-4e08-88cf-26cb9bc884ee None admin] RBAC: Authorizing `identity:get_user(user_id=2124c4c0223a4de18cdafb58d19473f8)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:26.612743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe868bdd-5791-4e08-88cf-26cb9bc884ee None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:26.614309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 538/1082] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:26 2022] GET /identity/v3/users/2124c4c0223a4de18cdafb58d19473f8 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:26.619981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e146dfb-bb09-440b-9203-bea324f5c04b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:26.716685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e146dfb-bb09-440b-9203-bea324f5c04b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:26.750453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b905d-6198-46fd-9d74-fde12f5064ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:26.751075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b905d-6198-46fd-9d74-fde12f5064ef None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:26.751354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b905d-6198-46fd-9d74-fde12f5064ef None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:26.751627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b905d-6198-46fd-9d74-fde12f5064ef None admin] PATH_INFO: `/v3/projects/d3d55145d5224f3d886c3d4c644de8f0` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:26.752778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b905d-6198-46fd-9d74-fde12f5064ef None admin] RBAC: Authorizing `identity:get_project(project_id=d3d55145d5224f3d886c3d4c644de8f0)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:26.754213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b905d-6198-46fd-9d74-fde12f5064ef None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:26.755851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 545/1083] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:18:26 2022] GET /identity/v3/projects/d3d55145d5224f3d886c3d4c644de8f0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:26.761914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe868bdd-5791-4e08-88cf-26cb9bc884ee None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:26.864083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe868bdd-5791-4e08-88cf-26cb9bc884ee None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:26.898994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8f411-f641-4e54-a309-57495a3f5b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:26.899645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8f411-f641-4e54-a309-57495a3f5b18 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:26.899920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8f411-f641-4e54-a309-57495a3f5b18 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:26.900189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8f411-f641-4e54-a309-57495a3f5b18 None admin] PATH_INFO: `/v3/projects/d3d55145d5224f3d886c3d4c644de8f0/users/2124c4c0223a4de18cdafb58d19473f8/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:26.902089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8f411-f641-4e54-a309-57495a3f5b18 None admin] RBAC: Authorizing `identity:create_grant(project_id=d3d55145d5224f3d886c3d4c644de8f0, user_id=2124c4c0223a4de18cdafb58d19473f8, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:26.903517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8f411-f641-4e54-a309-57495a3f5b18 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:26.916561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 539/1084] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:18:26 2022] PUT /identity/v3/projects/d3d55145d5224f3d886c3d4c644de8f0/users/2124c4c0223a4de18cdafb58d19473f8/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:18:27.675883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc9250-4894-4dc1-8cea-716ef89c8d9d None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:27.676413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc9250-4894-4dc1-8cea-716ef89c8d9d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:27.676561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc9250-4894-4dc1-8cea-716ef89c8d9d None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:27.677915 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 546/1085] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:27.683081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce12e5-f411-4f3f-8fe1-b391ede905be None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:27.683459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce12e5-f411-4f3f-8fe1-b391ede905be None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:27.683638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce12e5-f411-4f3f-8fe1-b391ede905be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:27.702492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-13ce12e5-f411-4f3f-8fe1-b391ede905be None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:27.704257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-13ce12e5-f411-4f3f-8fe1-b391ede905be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:27.747170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:27.776104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 540/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:28.053627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc9250-4894-4dc1-8cea-716ef89c8d9d None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:28.085660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:28.155499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc9250-4894-4dc1-8cea-716ef89c8d9d None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:28.189832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d292d-ddff-484e-8eaf-e998612dd86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:28.190451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d292d-ddff-484e-8eaf-e998612dd86d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:28.190726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d292d-ddff-484e-8eaf-e998612dd86d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:28.190998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d292d-ddff-484e-8eaf-e998612dd86d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:28.200860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d292d-ddff-484e-8eaf-e998612dd86d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-d26d292d-ddff-484e-8eaf-e998612dd86d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:28.206177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:28.207253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:28.208084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 547/1087] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 1 00:18:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:28.213280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce12e5-f411-4f3f-8fe1-b391ede905be None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:28.311543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce12e5-f411-4f3f-8fe1-b391ede905be None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:28.345071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6be4db-7a16-4802-9c23-b8928b17be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:28.345674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6be4db-7a16-4802-9c23-b8928b17be15 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:28.345954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6be4db-7a16-4802-9c23-b8928b17be15 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:28.346232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6be4db-7a16-4802-9c23-b8928b17be15 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:28.346893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6be4db-7a16-4802-9c23-b8928b17be15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:28.347195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6be4db-7a16-4802-9c23-b8928b17be15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:28.348581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6be4db-7a16-4802-9c23-b8928b17be15 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:28.356788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 541/1088] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 00:18:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:28.363012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d292d-ddff-484e-8eaf-e998612dd86d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:28.453137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:28.461961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d292d-ddff-484e-8eaf-e998612dd86d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:28.495459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9f634-3d2f-4835-9e31-0c87f4700902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:28.496083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9f634-3d2f-4835-9e31-0c87f4700902 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:28.496355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9f634-3d2f-4835-9e31-0c87f4700902 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:28.496659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9f634-3d2f-4835-9e31-0c87f4700902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:28.506984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9f634-3d2f-4835-9e31-0c87f4700902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:28.508418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9f634-3d2f-4835-9e31-0c87f4700902 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-97f9f634-3d2f-4835-9e31-0c87f4700902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:28.522933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:28.524062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:28.524976 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:28.524976 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:28.524976 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:28.524976 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 548/1089] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:28.529877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6be4db-7a16-4802-9c23-b8928b17be15 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:28.565250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:28.629061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6be4db-7a16-4802-9c23-b8928b17be15 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:28.661519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7592df-5719-4731-9723-d9254bad95cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:28.662120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7592df-5719-4731-9723-d9254bad95cf None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:28.662397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7592df-5719-4731-9723-d9254bad95cf None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:28.662663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7592df-5719-4731-9723-d9254bad95cf None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:28.663296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7592df-5719-4731-9723-d9254bad95cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:28.663595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7592df-5719-4731-9723-d9254bad95cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:28.664990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7592df-5719-4731-9723-d9254bad95cf None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:28.677143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 542/1090] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:28.682834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9f634-3d2f-4835-9e31-0c87f4700902 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:28.780628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9f634-3d2f-4835-9e31-0c87f4700902 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:28.803128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac4484-753f-41a8-a006-ce144d532a96 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:28.803631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac4484-753f-41a8-a006-ce144d532a96 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:28.803755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac4484-753f-41a8-a006-ce144d532a96 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:28.805057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 543/1091] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:28.810012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3018de-1bd4-453b-aa0f-21b3dce5ef04 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:28.810295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3018de-1bd4-453b-aa0f-21b3dce5ef04 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:28.810569 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3018de-1bd4-453b-aa0f-21b3dce5ef04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:28.814243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb0ab1-4610-4de9-bd13-635323a7de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:28.814863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb0ab1-4610-4de9-bd13-635323a7de31 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:28.815136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb0ab1-4610-4de9-bd13-635323a7de31 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:28.815445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb0ab1-4610-4de9-bd13-635323a7de31 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:28.822998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb0ab1-4610-4de9-bd13-635323a7de31 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:28.824432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb0ab1-4610-4de9-bd13-635323a7de31 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:28.828427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-8c3018de-1bd4-453b-aa0f-21b3dce5ef04 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:28.829634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8c3018de-1bd4-453b-aa0f-21b3dce5ef04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-f0bb0ab1-4610-4de9-bd13-635323a7de31 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:18:28.835501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:28.836457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:28.837908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 549/1092] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 1 00:18:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:28.842341 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb0ab1-4610-4de9-bd13-635323a7de31 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:28.847386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:28.896037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 544/1093] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:28.943665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb0ab1-4610-4de9-bd13-635323a7de31 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:28.978355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb45f-7347-435c-a449-c3065084899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:28.978956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb45f-7347-435c-a449-c3065084899c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:28.979303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb45f-7347-435c-a449-c3065084899c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:28.979577 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833eb45f-7347-435c-a449-c3065084899c None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:28.980342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eb45f-7347-435c-a449-c3065084899c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:28.980705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eb45f-7347-435c-a449-c3065084899c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:28.982240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833eb45f-7347-435c-a449-c3065084899c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:28.999622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 550/1094] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 1 00:18:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:29.005353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3018de-1bd4-453b-aa0f-21b3dce5ef04 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.032781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:29.111358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3018de-1bd4-453b-aa0f-21b3dce5ef04 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:29.144926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba815a-9f0b-44dd-b2e3-36be87d57df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:29.145528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba815a-9f0b-44dd-b2e3-36be87d57df6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:29.145806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba815a-9f0b-44dd-b2e3-36be87d57df6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:29.146074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba815a-9f0b-44dd-b2e3-36be87d57df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:29.164435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb45f-7347-435c-a449-c3065084899c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.212483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba815a-9f0b-44dd-b2e3-36be87d57df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:29.214027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba815a-9f0b-44dd-b2e3-36be87d57df6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:29.227696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:29.240467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:29.269218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833eb45f-7347-435c-a449-c3065084899c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:29.303987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f4b2-b473-41ee-8e59-a62dc145cf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:29.304630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f4b2-b473-41ee-8e59-a62dc145cf78 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:29.304908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f4b2-b473-41ee-8e59-a62dc145cf78 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:29.305178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f4b2-b473-41ee-8e59-a62dc145cf78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:29.312807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 545/1095] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:18:29.318616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba815a-9f0b-44dd-b2e3-36be87d57df6 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.319473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732f4b2-b473-41ee-8e59-a62dc145cf78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-7732f4b2-b473-41ee-8e59-a62dc145cf78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:29.324225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:29.325365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:29.326803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 551/1096] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:29.331845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f4b2-b473-41ee-8e59-a62dc145cf78 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.416043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba815a-9f0b-44dd-b2e3-36be87d57df6 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:29.430629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f4b2-b473-41ee-8e59-a62dc145cf78 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:29.441053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:29.450155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7465cd-19ab-40e6-be08-16f2abee8e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:29.450784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7465cd-19ab-40e6-be08-16f2abee8e36 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:29.451065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7465cd-19ab-40e6-be08-16f2abee8e36 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:29.451336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7465cd-19ab-40e6-be08-16f2abee8e36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:29.454925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:29.461918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7465cd-19ab-40e6-be08-16f2abee8e36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:29.463373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7465cd-19ab-40e6-be08-16f2abee8e36 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:29.467116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd677a-be19-4259-8622-ec89671df323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:29.467730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd677a-be19-4259-8622-ec89671df323 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:29.468108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd677a-be19-4259-8622-ec89671df323 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:29.468378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd677a-be19-4259-8622-ec89671df323 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:29.469117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd677a-be19-4259-8622-ec89671df323 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:29.469410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd677a-be19-4259-8622-ec89671df323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:29.470763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd677a-be19-4259-8622-ec89671df323 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-9d7465cd-19ab-40e6-be08-16f2abee8e36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:29.478531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:29.479560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:18:29.480461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:29.480461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:18:29.480461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:29.480675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 552/1097] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:29.480746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 546/1098] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:29.485046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd677a-be19-4259-8622-ec89671df323 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.485164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7465cd-19ab-40e6-be08-16f2abee8e36 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.584155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7465cd-19ab-40e6-be08-16f2abee8e36 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:29.585728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd677a-be19-4259-8622-ec89671df323 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:29.618635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1cb4e-0653-4915-b825-a01bf8c85940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:29.619240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1cb4e-0653-4915-b825-a01bf8c85940 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:29.619520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1cb4e-0653-4915-b825-a01bf8c85940 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:29.619817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1cb4e-0653-4915-b825-a01bf8c85940 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:29.620410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa921b1e-c066-423c-b72b-21e6182dbd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:29.620555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1cb4e-0653-4915-b825-a01bf8c85940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:29.620737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1cb4e-0653-4915-b825-a01bf8c85940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:29.621064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa921b1e-c066-423c-b72b-21e6182dbd39 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:29.621342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa921b1e-c066-423c-b72b-21e6182dbd39 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:29.621614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa921b1e-c066-423c-b72b-21e6182dbd39 None admin] PATH_INFO: `/v3/users/2124c4c0223a4de18cdafb58d19473f8` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:29.622134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1cb4e-0653-4915-b825-a01bf8c85940 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:29.622740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa921b1e-c066-423c-b72b-21e6182dbd39 None admin] RBAC: Authorizing `identity:get_user(user_id=2124c4c0223a4de18cdafb58d19473f8)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:29.624158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa921b1e-c066-423c-b72b-21e6182dbd39 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:29.625909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 553/1099] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/users/2124c4c0223a4de18cdafb58d19473f8 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:29.631658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa921b1e-c066-423c-b72b-21e6182dbd39 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.640145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 547/1100] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:29.641219 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1cb4e-0653-4915-b825-a01bf8c85940 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.681980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:29.691259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:29.739982 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa921b1e-c066-423c-b72b-21e6182dbd39 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:29.745556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1cb4e-0653-4915-b825-a01bf8c85940 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:29.773735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030c37f-5c81-4cdd-aa9a-045e2d8d2641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:29.774342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030c37f-5c81-4cdd-aa9a-045e2d8d2641 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:29.774622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030c37f-5c81-4cdd-aa9a-045e2d8d2641 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:29.774903 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030c37f-5c81-4cdd-aa9a-045e2d8d2641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:29.779623 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb64a36-34cb-4cb7-90a7-f9c943fc9b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:29.780248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb64a36-34cb-4cb7-90a7-f9c943fc9b6a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:29.780560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb64a36-34cb-4cb7-90a7-f9c943fc9b6a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:29.780839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb64a36-34cb-4cb7-90a7-f9c943fc9b6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:29.788077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb64a36-34cb-4cb7-90a7-f9c943fc9b6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:29.789599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb64a36-34cb-4cb7-90a7-f9c943fc9b6a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-7cb64a36-34cb-4cb7-90a7-f9c943fc9b6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:18:29.801926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:29.802811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:29.804130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:18:29.804130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:18:29.804130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:18:29.804130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:18:29.804130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:18:29.804130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:18:29.804130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:18:29.804130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:29.804130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 548/1101] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:18:29.808685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb64a36-34cb-4cb7-90a7-f9c943fc9b6a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.842850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030c37f-5c81-4cdd-aa9a-045e2d8d2641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:29.845665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030c37f-5c81-4cdd-aa9a-045e2d8d2641 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:29.897475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:29.905707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:29.909708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb64a36-34cb-4cb7-90a7-f9c943fc9b6a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:29.943519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93c236-9f28-435d-852f-facd7576b8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:29.944045 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93c236-9f28-435d-852f-facd7576b8b8 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:29.944320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93c236-9f28-435d-852f-facd7576b8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:29.944618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93c236-9f28-435d-852f-facd7576b8b8 None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:29.945300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93c236-9f28-435d-852f-facd7576b8b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:29.945600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93c236-9f28-435d-852f-facd7576b8b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:29.945944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 554/1102] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 00:18:29.947046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93c236-9f28-435d-852f-facd7576b8b8 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:29.951774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030c37f-5c81-4cdd-aa9a-045e2d8d2641 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:29.959106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 549/1103] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:29.964869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93c236-9f28-435d-852f-facd7576b8b8 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:30.058049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030c37f-5c81-4cdd-aa9a-045e2d8d2641 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:30.067214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93c236-9f28-435d-852f-facd7576b8b8 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:30.092468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c481923-f3b8-4cd8-a11a-d5c01ae8faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:30.093117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c481923-f3b8-4cd8-a11a-d5c01ae8faf6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:30.093896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c481923-f3b8-4cd8-a11a-d5c01ae8faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:30.094203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c481923-f3b8-4cd8-a11a-d5c01ae8faf6 None admin] PATH_INFO: `/v3/projects/d3d55145d5224f3d886c3d4c644de8f0` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:30.095294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c481923-f3b8-4cd8-a11a-d5c01ae8faf6 None admin] RBAC: Authorizing `identity:get_project(project_id=d3d55145d5224f3d886c3d4c644de8f0)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:30.096744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c481923-f3b8-4cd8-a11a-d5c01ae8faf6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:30.098352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 555/1104] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/projects/d3d55145d5224f3d886c3d4c644de8f0 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:30.101356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d131c1e-0868-4570-ae3d-0e318a6f7de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:30.101956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d131c1e-0868-4570-ae3d-0e318a6f7de6 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:30.102230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d131c1e-0868-4570-ae3d-0e318a6f7de6 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:30.102497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d131c1e-0868-4570-ae3d-0e318a6f7de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:30.103210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d131c1e-0868-4570-ae3d-0e318a6f7de6 None admin] RBAC: Adding query filter params (role.id=448c7e73924b47a1b6af2135604b5dbb, scope.project.id=ed93469862874c75951d95e28929c904, user.id=fb204040ab2f4168b35c3e78bacb849f) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:30.103609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d131c1e-0868-4570-ae3d-0e318a6f7de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:30.104551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c481923-f3b8-4cd8-a11a-d5c01ae8faf6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:30.105059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d131c1e-0868-4570-ae3d-0e318a6f7de6 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:30.114758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 550/1105] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:29 2022] GET /identity/v3/role_assignments?user.id=fb204040ab2f4168b35c3e78bacb849f&scope.project.id=ed93469862874c75951d95e28929c904&role.id=448c7e73924b47a1b6af2135604b5dbb => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:30.129291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:30.215806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c481923-f3b8-4cd8-a11a-d5c01ae8faf6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:30.253464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328241f5-6ec1-43f6-bbd3-c6ad54793327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:30.254074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328241f5-6ec1-43f6-bbd3-c6ad54793327 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:30.254353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328241f5-6ec1-43f6-bbd3-c6ad54793327 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:30.254624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328241f5-6ec1-43f6-bbd3-c6ad54793327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:30.255357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328241f5-6ec1-43f6-bbd3-c6ad54793327 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=d3d55145d5224f3d886c3d4c644de8f0, user.id=2124c4c0223a4de18cdafb58d19473f8) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:30.255668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328241f5-6ec1-43f6-bbd3-c6ad54793327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:30.257622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328241f5-6ec1-43f6-bbd3-c6ad54793327 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:30.263285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 556/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:30 2022] GET /identity/v3/role_assignments?user.id=2124c4c0223a4de18cdafb58d19473f8&scope.project.id=d3d55145d5224f3d886c3d4c644de8f0&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:32.189004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7be4a4-553c-4066-930c-aa30c3d13cef None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:32.189564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7be4a4-553c-4066-930c-aa30c3d13cef None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:32.189564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7be4a4-553c-4066-930c-aa30c3d13cef None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:32.190896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 551/1107] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:32.196025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66265198-9116-4c91-bb4b-952e545d683a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:32.196431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66265198-9116-4c91-bb4b-952e545d683a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:32.196595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66265198-9116-4c91-bb4b-952e545d683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:32.215037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-66265198-9116-4c91-bb4b-952e545d683a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:32.216210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-66265198-9116-4c91-bb4b-952e545d683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:32.283288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 557/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:32.554202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7be4a4-553c-4066-930c-aa30c3d13cef None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:32.578768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:32.655432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7be4a4-553c-4066-930c-aa30c3d13cef None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:32.688795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6778635-f3e8-4dc2-93d6-e93214f85f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:32.689413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6778635-f3e8-4dc2-93d6-e93214f85f20 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:32.689696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6778635-f3e8-4dc2-93d6-e93214f85f20 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:32.689969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6778635-f3e8-4dc2-93d6-e93214f85f20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:32.691071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6778635-f3e8-4dc2-93d6-e93214f85f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:32.692542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6778635-f3e8-4dc2-93d6-e93214f85f20 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:32.694104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 552/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:32.699829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66265198-9116-4c91-bb4b-952e545d683a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:32.714996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:32.797564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66265198-9116-4c91-bb4b-952e545d683a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:32.830267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb6bd2-ec22-4fa4-baa7-1c542bc0788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:32.830801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb6bd2-ec22-4fa4-baa7-1c542bc0788e None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:32.831074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb6bd2-ec22-4fa4-baa7-1c542bc0788e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:32.831347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb6bd2-ec22-4fa4-baa7-1c542bc0788e None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:32.831996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb6bd2-ec22-4fa4-baa7-1c542bc0788e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:32.833444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb6bd2-ec22-4fa4-baa7-1c542bc0788e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:32.854412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 558/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:18:32 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:34.794900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9c48d-59e6-405c-a617-661b3131532f None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:34.795427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9c48d-59e6-405c-a617-661b3131532f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:34.795427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9c48d-59e6-405c-a617-661b3131532f None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:34.796850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 553/1111] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:34.802060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548f251-60bc-40bf-917c-ebe18e23ae63 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:34.802494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548f251-60bc-40bf-917c-ebe18e23ae63 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:34.802616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548f251-60bc-40bf-917c-ebe18e23ae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:34.820932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-1548f251-60bc-40bf-917c-ebe18e23ae63 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:34.822109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1548f251-60bc-40bf-917c-ebe18e23ae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:34.883112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:34.888116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 559/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:35.156950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9c48d-59e6-405c-a617-661b3131532f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:35.245882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:35.257604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9c48d-59e6-405c-a617-661b3131532f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:35.290439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c783e-997d-4184-b418-bdb856f09abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:35.291053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c783e-997d-4184-b418-bdb856f09abd None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:35.291331 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c783e-997d-4184-b418-bdb856f09abd None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:35.291602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c783e-997d-4184-b418-bdb856f09abd None admin] PATH_INFO: `/v3/roles/2986c98e353a489592c69ada5c3c2aaf` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:35.293000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c783e-997d-4184-b418-bdb856f09abd None admin] RBAC: Authorizing `identity:get_role(role_id=2986c98e353a489592c69ada5c3c2aaf)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:35.294377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c783e-997d-4184-b418-bdb856f09abd None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:35.295697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 554/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:35 2022] GET /identity/v3/roles/2986c98e353a489592c69ada5c3c2aaf => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:35.302154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548f251-60bc-40bf-917c-ebe18e23ae63 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:35.398311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548f251-60bc-40bf-917c-ebe18e23ae63 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:35.431949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a7f01-d2cd-49f7-8180-c702d36ef3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:35.432530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a7f01-d2cd-49f7-8180-c702d36ef3f4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:35.432813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a7f01-d2cd-49f7-8180-c702d36ef3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:35.433084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a7f01-d2cd-49f7-8180-c702d36ef3f4 None admin] PATH_INFO: `/v3/users/71f6516053c04c31aaf13cc7f633b0d0` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:35.445727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a7f01-d2cd-49f7-8180-c702d36ef3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=71f6516053c04c31aaf13cc7f633b0d0)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:35.447180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a7f01-d2cd-49f7-8180-c702d36ef3f4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:35.448781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 560/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:35 2022] GET /identity/v3/users/71f6516053c04c31aaf13cc7f633b0d0 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:35.454426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c783e-997d-4184-b418-bdb856f09abd None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:35.551786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c783e-997d-4184-b418-bdb856f09abd None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:35.584377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157dbfc5-77c7-4760-9a72-2afcee121f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:35.585052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dbfc5-77c7-4760-9a72-2afcee121f8d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:35.585328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dbfc5-77c7-4760-9a72-2afcee121f8d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:35.585597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157dbfc5-77c7-4760-9a72-2afcee121f8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:35.610938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:35.650555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157dbfc5-77c7-4760-9a72-2afcee121f8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:35.652005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157dbfc5-77c7-4760-9a72-2afcee121f8d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:35.746957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 555/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:18:35.753235 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a7f01-d2cd-49f7-8180-c702d36ef3f4 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:35.813897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:35.853759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a7f01-d2cd-49f7-8180-c702d36ef3f4 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:35.887404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222422b-dcbf-4c2b-b016-ce1611be6073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:35.888023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222422b-dcbf-4c2b-b016-ce1611be6073 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:35.888298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222422b-dcbf-4c2b-b016-ce1611be6073 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:35.888594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222422b-dcbf-4c2b-b016-ce1611be6073 None admin] PATH_INFO: `/v3/projects/d3d55145d5224f3d886c3d4c644de8f0` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:35.889690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222422b-dcbf-4c2b-b016-ce1611be6073 None admin] RBAC: Authorizing `identity:get_project(project_id=d3d55145d5224f3d886c3d4c644de8f0)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:35.891330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222422b-dcbf-4c2b-b016-ce1611be6073 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:35.892960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 561/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:18:35 2022] GET /identity/v3/projects/d3d55145d5224f3d886c3d4c644de8f0 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:35.899081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157dbfc5-77c7-4760-9a72-2afcee121f8d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:35.956266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:35.997179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157dbfc5-77c7-4760-9a72-2afcee121f8d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:36.029858 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd050a3-f8fc-4dbe-a43f-098e6c1b9ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:36.030452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd050a3-f8fc-4dbe-a43f-098e6c1b9ee0 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:36.030759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd050a3-f8fc-4dbe-a43f-098e6c1b9ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:36.031033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd050a3-f8fc-4dbe-a43f-098e6c1b9ee0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:36.031742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd050a3-f8fc-4dbe-a43f-098e6c1b9ee0 None admin] RBAC: Adding query filter params (role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=d3d55145d5224f3d886c3d4c644de8f0, user.id=71f6516053c04c31aaf13cc7f633b0d0) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:36.032047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd050a3-f8fc-4dbe-a43f-098e6c1b9ee0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:36.033499 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd050a3-f8fc-4dbe-a43f-098e6c1b9ee0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:36.038682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 556/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:35 2022] GET /identity/v3/role_assignments?user.id=71f6516053c04c31aaf13cc7f633b0d0&scope.project.id=d3d55145d5224f3d886c3d4c644de8f0&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:37.922967 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20e6f5-7708-49b9-822f-59f06163b019 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:37.923568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20e6f5-7708-49b9-822f-59f06163b019 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:37.923568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20e6f5-7708-49b9-822f-59f06163b019 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:37.924964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 562/1118] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:18:37.930117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4ae33-7ba2-4690-b162-3c42bac60319 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:37.930560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4ae33-7ba2-4690-b162-3c42bac60319 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:37.930660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4ae33-7ba2-4690-b162-3c42bac60319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:37.949399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-f9f4ae33-7ba2-4690-b162-3c42bac60319 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:37.950564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f9f4ae33-7ba2-4690-b162-3c42bac60319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:38.018647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 557/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 00:18:38.284601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20e6f5-7708-49b9-822f-59f06163b019 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:38.386546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20e6f5-7708-49b9-822f-59f06163b019 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:38.412910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:38.422563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9900d3f-5ac5-45c7-99a5-020cc80a6fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:38.423185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9900d3f-5ac5-45c7-99a5-020cc80a6fe9 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:38.423461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9900d3f-5ac5-45c7-99a5-020cc80a6fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:38.423730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9900d3f-5ac5-45c7-99a5-020cc80a6fe9 None admin] PATH_INFO: `/v3/roles/2986c98e353a489592c69ada5c3c2aaf` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:38.425146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9900d3f-5ac5-45c7-99a5-020cc80a6fe9 None admin] RBAC: Authorizing `identity:get_role(role_id=2986c98e353a489592c69ada5c3c2aaf)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:38.426514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9900d3f-5ac5-45c7-99a5-020cc80a6fe9 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:38.427830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 563/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:38 2022] GET /identity/v3/roles/2986c98e353a489592c69ada5c3c2aaf => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:38.433622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4ae33-7ba2-4690-b162-3c42bac60319 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:38.485411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:38.529494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4ae33-7ba2-4690-b162-3c42bac60319 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:38.562504 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca23751-ae73-400d-b556-b0bd850fff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:38.563114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca23751-ae73-400d-b556-b0bd850fff3f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:38.563387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca23751-ae73-400d-b556-b0bd850fff3f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:38.563655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca23751-ae73-400d-b556-b0bd850fff3f None admin] PATH_INFO: `/v3/users/71f6516053c04c31aaf13cc7f633b0d0` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:38.564774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca23751-ae73-400d-b556-b0bd850fff3f None admin] RBAC: Authorizing `identity:get_user(user_id=71f6516053c04c31aaf13cc7f633b0d0)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:38.566202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca23751-ae73-400d-b556-b0bd850fff3f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:38.567740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 558/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:38 2022] GET /identity/v3/users/71f6516053c04c31aaf13cc7f633b0d0 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:38.573293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9900d3f-5ac5-45c7-99a5-020cc80a6fe9 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:38.670661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9900d3f-5ac5-45c7-99a5-020cc80a6fe9 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:38.704662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b33e69-594e-4d1c-9c08-c68e61181968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:38.705276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b33e69-594e-4d1c-9c08-c68e61181968 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:38.705552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b33e69-594e-4d1c-9c08-c68e61181968 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:38.705823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b33e69-594e-4d1c-9c08-c68e61181968 None admin] PATH_INFO: `/v3/projects/d3d55145d5224f3d886c3d4c644de8f0` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:38.706917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b33e69-594e-4d1c-9c08-c68e61181968 None admin] RBAC: Authorizing `identity:get_project(project_id=d3d55145d5224f3d886c3d4c644de8f0)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:38.708347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b33e69-594e-4d1c-9c08-c68e61181968 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:38.709986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 564/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:18:38 2022] GET /identity/v3/projects/d3d55145d5224f3d886c3d4c644de8f0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:38.715879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca23751-ae73-400d-b556-b0bd850fff3f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:38.813916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca23751-ae73-400d-b556-b0bd850fff3f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:38.832914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:38.847190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54436b7-b96f-4b13-988e-535871f9dfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:38.847840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54436b7-b96f-4b13-988e-535871f9dfd0 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:38.848118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54436b7-b96f-4b13-988e-535871f9dfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:38.848387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54436b7-b96f-4b13-988e-535871f9dfd0 None admin] PATH_INFO: `/v3/projects/d3d55145d5224f3d886c3d4c644de8f0/users/71f6516053c04c31aaf13cc7f633b0d0/roles/2986c98e353a489592c69ada5c3c2aaf` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:38.850799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54436b7-b96f-4b13-988e-535871f9dfd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d3d55145d5224f3d886c3d4c644de8f0, user_id=71f6516053c04c31aaf13cc7f633b0d0, role_id=2986c98e353a489592c69ada5c3c2aaf)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:38.852257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54436b7-b96f-4b13-988e-535871f9dfd0 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:38.865851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 559/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:18:38 2022] PUT /identity/v3/projects/d3d55145d5224f3d886c3d4c644de8f0/users/71f6516053c04c31aaf13cc7f633b0d0/roles/2986c98e353a489592c69ada5c3c2aaf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:18:40.735860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ccac5-dc6c-46bb-89b6-fbac5e471b76 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:40.736452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ccac5-dc6c-46bb-89b6-fbac5e471b76 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:40.736452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ccac5-dc6c-46bb-89b6-fbac5e471b76 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:40.737760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 565/1124] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:40.742731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891ad65-be6a-4b04-81ab-0e3f1a4f8271 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:40.743094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891ad65-be6a-4b04-81ab-0e3f1a4f8271 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:40.743276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891ad65-be6a-4b04-81ab-0e3f1a4f8271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:40.762142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1891ad65-be6a-4b04-81ab-0e3f1a4f8271 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:40.763330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1891ad65-be6a-4b04-81ab-0e3f1a4f8271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:40.828700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 560/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:41.095290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ccac5-dc6c-46bb-89b6-fbac5e471b76 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:41.156467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:41.195700 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ccac5-dc6c-46bb-89b6-fbac5e471b76 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:41.227891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3146c-b23e-4c2c-b672-61af35ec7893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:41.228539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3146c-b23e-4c2c-b672-61af35ec7893 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:41.228805 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3146c-b23e-4c2c-b672-61af35ec7893 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:41.229075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3146c-b23e-4c2c-b672-61af35ec7893 None admin] PATH_INFO: `/v3/roles/2986c98e353a489592c69ada5c3c2aaf` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:41.230460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3146c-b23e-4c2c-b672-61af35ec7893 None admin] RBAC: Authorizing `identity:get_role(role_id=2986c98e353a489592c69ada5c3c2aaf)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:41.231830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3146c-b23e-4c2c-b672-61af35ec7893 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:41.233157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 566/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:41 2022] GET /identity/v3/roles/2986c98e353a489592c69ada5c3c2aaf => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:41.239372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891ad65-be6a-4b04-81ab-0e3f1a4f8271 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:41.335285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891ad65-be6a-4b04-81ab-0e3f1a4f8271 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:41.346482 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:41.370537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bdcbd-9dd2-43c2-bfb2-88fe814cd4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:41.371249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bdcbd-9dd2-43c2-bfb2-88fe814cd4e9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:41.371535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bdcbd-9dd2-43c2-bfb2-88fe814cd4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:41.371812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bdcbd-9dd2-43c2-bfb2-88fe814cd4e9 None admin] PATH_INFO: `/v3/users/71f6516053c04c31aaf13cc7f633b0d0` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:41.372955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bdcbd-9dd2-43c2-bfb2-88fe814cd4e9 None admin] RBAC: Authorizing `identity:get_user(user_id=71f6516053c04c31aaf13cc7f633b0d0)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:41.374538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bdcbd-9dd2-43c2-bfb2-88fe814cd4e9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:41.376113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 561/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:41 2022] GET /identity/v3/users/71f6516053c04c31aaf13cc7f633b0d0 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:41.381795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3146c-b23e-4c2c-b672-61af35ec7893 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:41.478072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3146c-b23e-4c2c-b672-61af35ec7893 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:41.510426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:41.516551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3f88d-4191-440a-9fad-85990714b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:41.517146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3f88d-4191-440a-9fad-85990714b5ff None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:41.517421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3f88d-4191-440a-9fad-85990714b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:41.517690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3f88d-4191-440a-9fad-85990714b5ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:41.581731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3f88d-4191-440a-9fad-85990714b5ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:41.583155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3f88d-4191-440a-9fad-85990714b5ff None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:41.680627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 567/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 00:18:41.686443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bdcbd-9dd2-43c2-bfb2-88fe814cd4e9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:41.792741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bdcbd-9dd2-43c2-bfb2-88fe814cd4e9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:41.825614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8f948-6cce-4fc6-84e3-221977170355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:41.826232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8f948-6cce-4fc6-84e3-221977170355 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:41.826511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8f948-6cce-4fc6-84e3-221977170355 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:41.826781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8f948-6cce-4fc6-84e3-221977170355 None admin] PATH_INFO: `/v3/projects/d3d55145d5224f3d886c3d4c644de8f0` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:41.828639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8f948-6cce-4fc6-84e3-221977170355 None admin] RBAC: Authorizing `identity:get_project(project_id=d3d55145d5224f3d886c3d4c644de8f0)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:41.830080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8f948-6cce-4fc6-84e3-221977170355 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:41.831691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 562/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:18:41 2022] GET /identity/v3/projects/d3d55145d5224f3d886c3d4c644de8f0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:41.837798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3f88d-4191-440a-9fad-85990714b5ff None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:41.867439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:41.934259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3f88d-4191-440a-9fad-85990714b5ff None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:41.966749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4dc38e-9340-4fbf-a29e-9635577d33e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:41.967329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dc38e-9340-4fbf-a29e-9635577d33e4 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:41.967601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dc38e-9340-4fbf-a29e-9635577d33e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:41.968519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4dc38e-9340-4fbf-a29e-9635577d33e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:41.969582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4dc38e-9340-4fbf-a29e-9635577d33e4 None admin] RBAC: Adding query filter params (role.id=2986c98e353a489592c69ada5c3c2aaf, scope.project.id=d3d55145d5224f3d886c3d4c644de8f0, user.id=71f6516053c04c31aaf13cc7f633b0d0) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:41.969886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4dc38e-9340-4fbf-a29e-9635577d33e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:41.971333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4dc38e-9340-4fbf-a29e-9635577d33e4 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:41.977167 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 568/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:41 2022] GET /identity/v3/role_assignments?user.id=71f6516053c04c31aaf13cc7f633b0d0&scope.project.id=d3d55145d5224f3d886c3d4c644de8f0&role.id=2986c98e353a489592c69ada5c3c2aaf => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:43.857009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdcc73-dbbb-4755-b756-c84cf222a113 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:43.857542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdcc73-dbbb-4755-b756-c84cf222a113 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:43.857542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdcc73-dbbb-4755-b756-c84cf222a113 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:43.859598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 563/1131] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:43.864921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28810c6c-b335-464a-a791-84a29cd6a9be None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:43.865344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28810c6c-b335-464a-a791-84a29cd6a9be None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:43.865468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28810c6c-b335-464a-a791-84a29cd6a9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:43.883176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-28810c6c-b335-464a-a791-84a29cd6a9be None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:43.884346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-28810c6c-b335-464a-a791-84a29cd6a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:43.948436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 569/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:44.219017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdcc73-dbbb-4755-b756-c84cf222a113 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:44.268318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:44.322907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdcc73-dbbb-4755-b756-c84cf222a113 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:44.355658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd33f12-1bae-4703-bee4-aa56f5bacc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:44.356264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd33f12-1bae-4703-bee4-aa56f5bacc19 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:44.356559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd33f12-1bae-4703-bee4-aa56f5bacc19 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:44.356829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd33f12-1bae-4703-bee4-aa56f5bacc19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:44.357912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd33f12-1bae-4703-bee4-aa56f5bacc19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:44.359907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd33f12-1bae-4703-bee4-aa56f5bacc19 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:44.361479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 564/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:44.367297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28810c6c-b335-464a-a791-84a29cd6a9be None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:44.391207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:44.463491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28810c6c-b335-464a-a791-84a29cd6a9be None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:44.497568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82484c-2dda-4084-8dd5-9e9935e9083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:44.498099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82484c-2dda-4084-8dd5-9e9935e9083a None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:44.498378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82484c-2dda-4084-8dd5-9e9935e9083a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:44.498647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82484c-2dda-4084-8dd5-9e9935e9083a None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:44.499420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82484c-2dda-4084-8dd5-9e9935e9083a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:44.500870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82484c-2dda-4084-8dd5-9e9935e9083a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:44.526029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 570/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:18:44 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:46.421659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90906df-1dd9-4839-acf0-1819560c0c37 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:46.422239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90906df-1dd9-4839-acf0-1819560c0c37 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:46.422239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90906df-1dd9-4839-acf0-1819560c0c37 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:46.423520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 565/1135] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:46.428601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c389a-85c1-48c5-9341-48702a8012ff None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:46.429015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c389a-85c1-48c5-9341-48702a8012ff None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:46.429150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c389a-85c1-48c5-9341-48702a8012ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:46.447410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-760c389a-85c1-48c5-9341-48702a8012ff None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:46.448616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-760c389a-85c1-48c5-9341-48702a8012ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:46.513059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 571/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:46.798081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90906df-1dd9-4839-acf0-1819560c0c37 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:46.895413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90906df-1dd9-4839-acf0-1819560c0c37 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:46.909256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:46.929040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bed5ce-3728-45ae-9e25-d92590c6be0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:46.929645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bed5ce-3728-45ae-9e25-d92590c6be0b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:46.929915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bed5ce-3728-45ae-9e25-d92590c6be0b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:46.930185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bed5ce-3728-45ae-9e25-d92590c6be0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:46.931270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bed5ce-3728-45ae-9e25-d92590c6be0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:46.932716 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bed5ce-3728-45ae-9e25-d92590c6be0b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:46.934245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 566/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:18:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:46.940181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c389a-85c1-48c5-9341-48702a8012ff None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:46.948266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:47.038501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c389a-85c1-48c5-9341-48702a8012ff None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:47.071267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa66cd-2c22-42e9-b7d3-01123e01f337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:47.071799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa66cd-2c22-42e9-b7d3-01123e01f337 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:47.072074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa66cd-2c22-42e9-b7d3-01123e01f337 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:47.072339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa66cd-2c22-42e9-b7d3-01123e01f337 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:47.073003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa66cd-2c22-42e9-b7d3-01123e01f337 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:47.074424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa66cd-2c22-42e9-b7d3-01123e01f337 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:47.096844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 572/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:18:46 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:18:49.005527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f60f90-4b44-4c69-bb79-51547b2cca95 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:49.006087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f60f90-4b44-4c69-bb79-51547b2cca95 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:49.006087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f60f90-4b44-4c69-bb79-51547b2cca95 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:49.007391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 567/1139] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:49.012407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f448980-ee48-47a0-924b-05972a30306b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:49.012814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f448980-ee48-47a0-924b-05972a30306b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:49.012986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f448980-ee48-47a0-924b-05972a30306b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:49.031408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0f448980-ee48-47a0-924b-05972a30306b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:49.032628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0f448980-ee48-47a0-924b-05972a30306b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:49.089314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:49.100790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 573/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:49.369690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f60f90-4b44-4c69-bb79-51547b2cca95 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:49.468966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f60f90-4b44-4c69-bb79-51547b2cca95 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:49.502201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2cf1-2f0e-410c-82d6-8e7c169ded87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:49.502810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2cf1-2f0e-410c-82d6-8e7c169ded87 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:49.503086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2cf1-2f0e-410c-82d6-8e7c169ded87 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:49.503354 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2cf1-2f0e-410c-82d6-8e7c169ded87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:49.513288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b2cf1-2f0e-410c-82d6-8e7c169ded87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-b54b2cf1-2f0e-410c-82d6-8e7c169ded87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:49.518152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:49.519284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:49.520223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 568/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:18:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:49.525407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f448980-ee48-47a0-924b-05972a30306b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:49.621633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f448980-ee48-47a0-924b-05972a30306b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:49.653430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef338d-feb5-4d2c-9b3b-63e8cd4871c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:49.654022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef338d-feb5-4d2c-9b3b-63e8cd4871c8 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:49.654295 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef338d-feb5-4d2c-9b3b-63e8cd4871c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:49.654564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef338d-feb5-4d2c-9b3b-63e8cd4871c8 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:49.655711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef338d-feb5-4d2c-9b3b-63e8cd4871c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:49.656004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef338d-feb5-4d2c-9b3b-63e8cd4871c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:49.657386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef338d-feb5-4d2c-9b3b-63e8cd4871c8 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:49.665067 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 574/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:18:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:49.672810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2cf1-2f0e-410c-82d6-8e7c169ded87 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:49.713899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:49.769298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2cf1-2f0e-410c-82d6-8e7c169ded87 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:49.802934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab18ea-89a1-4ad3-9d6f-0f164a6bf3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:49.803545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab18ea-89a1-4ad3-9d6f-0f164a6bf3e9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:49.803819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab18ea-89a1-4ad3-9d6f-0f164a6bf3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:49.804087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab18ea-89a1-4ad3-9d6f-0f164a6bf3e9 None admin] PATH_INFO: `/v3/users/a84fa877cab7409b968ab1f7d83e1886` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:49.817447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab18ea-89a1-4ad3-9d6f-0f164a6bf3e9 None admin] RBAC: Authorizing `identity:get_user(user_id=a84fa877cab7409b968ab1f7d83e1886)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:49.819939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab18ea-89a1-4ad3-9d6f-0f164a6bf3e9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:49.822116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 569/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:49 2022] GET /identity/v3/users/a84fa877cab7409b968ab1f7d83e1886 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:49.828387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef338d-feb5-4d2c-9b3b-63e8cd4871c8 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:49.883440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:49.927432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef338d-feb5-4d2c-9b3b-63e8cd4871c8 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:49.960001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd25b0-85db-4b07-a806-853856004261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:49.960619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd25b0-85db-4b07-a806-853856004261 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:49.960896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd25b0-85db-4b07-a806-853856004261 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:49.961163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd25b0-85db-4b07-a806-853856004261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:50.026494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd25b0-85db-4b07-a806-853856004261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:50.027918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd25b0-85db-4b07-a806-853856004261 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:50.079300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:50.123435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 575/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:18:50.129545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab18ea-89a1-4ad3-9d6f-0f164a6bf3e9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:50.223802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:50.226838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab18ea-89a1-4ad3-9d6f-0f164a6bf3e9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:50.260445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a9ded-2704-4ac3-943e-91f0c12044d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:50.261090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a9ded-2704-4ac3-943e-91f0c12044d3 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:50.261363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a9ded-2704-4ac3-943e-91f0c12044d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:50.261632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a9ded-2704-4ac3-943e-91f0c12044d3 None admin] PATH_INFO: `/v3/projects/6ae4dbb2b8aa473c9833ee83e7a96aef` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:50.262723 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a9ded-2704-4ac3-943e-91f0c12044d3 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae4dbb2b8aa473c9833ee83e7a96aef)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:50.264717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a9ded-2704-4ac3-943e-91f0c12044d3 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:50.266555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 570/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:18:50 2022] GET /identity/v3/projects/6ae4dbb2b8aa473c9833ee83e7a96aef => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:50.272838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd25b0-85db-4b07-a806-853856004261 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:50.370246 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd25b0-85db-4b07-a806-853856004261 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:50.403389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b044cd9-3032-41f1-a148-c0309331294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:50.403977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b044cd9-3032-41f1-a148-c0309331294d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:50.404253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b044cd9-3032-41f1-a148-c0309331294d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:50.404548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b044cd9-3032-41f1-a148-c0309331294d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:50.405898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b044cd9-3032-41f1-a148-c0309331294d None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=6ae4dbb2b8aa473c9833ee83e7a96aef, user.id=a84fa877cab7409b968ab1f7d83e1886) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:50.406198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b044cd9-3032-41f1-a148-c0309331294d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:50.407637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b044cd9-3032-41f1-a148-c0309331294d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:50.412901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 576/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:50 2022] GET /identity/v3/role_assignments?user.id=a84fa877cab7409b968ab1f7d83e1886&scope.project.id=6ae4dbb2b8aa473c9833ee83e7a96aef&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:52.295356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c19e0-4662-41ee-b2ca-6f152ae642d1 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:52.296114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c19e0-4662-41ee-b2ca-6f152ae642d1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:52.296114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c19e0-4662-41ee-b2ca-6f152ae642d1 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:52.297314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 571/1147] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:52.302464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbe0c7-bbdf-427e-ab66-d6afb5241a36 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:52.302846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbe0c7-bbdf-427e-ab66-d6afb5241a36 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:52.303005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbe0c7-bbdf-427e-ab66-d6afb5241a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:52.321462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-cadbe0c7-bbdf-427e-ab66-d6afb5241a36 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:52.322627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-cadbe0c7-bbdf-427e-ab66-d6afb5241a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:52.335689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:52.388838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 577/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:52.655722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c19e0-4662-41ee-b2ca-6f152ae642d1 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:52.767684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c19e0-4662-41ee-b2ca-6f152ae642d1 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:52.800335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961582a3-ace0-4421-8e0c-5d763b2e001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:52.800986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961582a3-ace0-4421-8e0c-5d763b2e001a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:52.801265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961582a3-ace0-4421-8e0c-5d763b2e001a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:52.801539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961582a3-ace0-4421-8e0c-5d763b2e001a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:52.812066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961582a3-ace0-4421-8e0c-5d763b2e001a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-961582a3-ace0-4421-8e0c-5d763b2e001a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:52.816883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:52.817940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:52.818970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 572/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:18:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:52.824091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbe0c7-bbdf-427e-ab66-d6afb5241a36 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:52.919595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbe0c7-bbdf-427e-ab66-d6afb5241a36 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:52.952389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d14556-bd8d-48c4-91e3-d4f07f8aa105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:52.953002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d14556-bd8d-48c4-91e3-d4f07f8aa105 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:52.953276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d14556-bd8d-48c4-91e3-d4f07f8aa105 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:52.953542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d14556-bd8d-48c4-91e3-d4f07f8aa105 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:52.954184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d14556-bd8d-48c4-91e3-d4f07f8aa105 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:52.954480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d14556-bd8d-48c4-91e3-d4f07f8aa105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:52.955835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d14556-bd8d-48c4-91e3-d4f07f8aa105 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:52.963530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 578/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:52.969692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961582a3-ace0-4421-8e0c-5d763b2e001a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:52.998147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:53.070753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961582a3-ace0-4421-8e0c-5d763b2e001a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:53.104207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d4160-438e-473e-b08d-5743f4b7d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:53.104852 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d4160-438e-473e-b08d-5743f4b7d593 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:53.105134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d4160-438e-473e-b08d-5743f4b7d593 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:53.105409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d4160-438e-473e-b08d-5743f4b7d593 None admin] PATH_INFO: `/v3/users/a84fa877cab7409b968ab1f7d83e1886` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:53.106523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d4160-438e-473e-b08d-5743f4b7d593 None admin] RBAC: Authorizing `identity:get_user(user_id=a84fa877cab7409b968ab1f7d83e1886)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:53.108517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d4160-438e-473e-b08d-5743f4b7d593 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:53.110288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 573/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:52 2022] GET /identity/v3/users/a84fa877cab7409b968ab1f7d83e1886 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:18:53.116146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d14556-bd8d-48c4-91e3-d4f07f8aa105 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:53.129455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:53.214256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d14556-bd8d-48c4-91e3-d4f07f8aa105 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:53.247142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eed82a-dd62-4d7c-a6b2-1092bfab14cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:53.247761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eed82a-dd62-4d7c-a6b2-1092bfab14cb None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:53.248041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eed82a-dd62-4d7c-a6b2-1092bfab14cb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:53.249276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eed82a-dd62-4d7c-a6b2-1092bfab14cb None admin] PATH_INFO: `/v3/projects/6ae4dbb2b8aa473c9833ee83e7a96aef` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:53.250462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eed82a-dd62-4d7c-a6b2-1092bfab14cb None admin] RBAC: Authorizing `identity:get_project(project_id=6ae4dbb2b8aa473c9833ee83e7a96aef)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:53.251900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eed82a-dd62-4d7c-a6b2-1092bfab14cb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:53.253554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 579/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:18:53 2022] GET /identity/v3/projects/6ae4dbb2b8aa473c9833ee83e7a96aef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:53.259397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d4160-438e-473e-b08d-5743f4b7d593 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:53.353197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:53.359971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d4160-438e-473e-b08d-5743f4b7d593 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:53.392463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2edc9-8903-482b-b803-8c80deb2059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:53.393134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2edc9-8903-482b-b803-8c80deb2059e None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:53.393409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2edc9-8903-482b-b803-8c80deb2059e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:53.393681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2edc9-8903-482b-b803-8c80deb2059e None admin] PATH_INFO: `/v3/projects/6ae4dbb2b8aa473c9833ee83e7a96aef/users/a84fa877cab7409b968ab1f7d83e1886/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:53.395546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2edc9-8903-482b-b803-8c80deb2059e None admin] RBAC: Authorizing `identity:create_grant(project_id=6ae4dbb2b8aa473c9833ee83e7a96aef, user_id=a84fa877cab7409b968ab1f7d83e1886, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:53.397016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2edc9-8903-482b-b803-8c80deb2059e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:53.410543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 574/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:18:53 2022] PUT /identity/v3/projects/6ae4dbb2b8aa473c9833ee83e7a96aef/users/a84fa877cab7409b968ab1f7d83e1886/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:18:55.298502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226d4b0-424b-40ed-98f4-2b96b8bfa557 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:55.299025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226d4b0-424b-40ed-98f4-2b96b8bfa557 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:55.299025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226d4b0-424b-40ed-98f4-2b96b8bfa557 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:55.300360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 580/1154] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:55.305364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321bfb-e50f-47c6-86bf-60b8821e9b03 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:55.305797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321bfb-e50f-47c6-86bf-60b8821e9b03 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:55.305917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321bfb-e50f-47c6-86bf-60b8821e9b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:55.324187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-14321bfb-e50f-47c6-86bf-60b8821e9b03 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:55.325442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-14321bfb-e50f-47c6-86bf-60b8821e9b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:55.390097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 575/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:55.653861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226d4b0-424b-40ed-98f4-2b96b8bfa557 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:55.730920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:55.752962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226d4b0-424b-40ed-98f4-2b96b8bfa557 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:55.787396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937cfc-b21c-4e42-8fa8-252302bffd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:55.788012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937cfc-b21c-4e42-8fa8-252302bffd43 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:55.788286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937cfc-b21c-4e42-8fa8-252302bffd43 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:55.788579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77937cfc-b21c-4e42-8fa8-252302bffd43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:55.799336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77937cfc-b21c-4e42-8fa8-252302bffd43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-77937cfc-b21c-4e42-8fa8-252302bffd43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:18:55.804071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:55.805195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:18:55.806046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 581/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:18:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:55.811408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321bfb-e50f-47c6-86bf-60b8821e9b03 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:55.893514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:55.909503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321bfb-e50f-47c6-86bf-60b8821e9b03 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:55.943032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7276d1-5cf5-4b4d-9675-1f26d26a9c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:55.943631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7276d1-5cf5-4b4d-9675-1f26d26a9c6b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:55.943909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7276d1-5cf5-4b4d-9675-1f26d26a9c6b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:55.944181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7276d1-5cf5-4b4d-9675-1f26d26a9c6b None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:55.944868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7276d1-5cf5-4b4d-9675-1f26d26a9c6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:55.945171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7276d1-5cf5-4b4d-9675-1f26d26a9c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:55.946537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7276d1-5cf5-4b4d-9675-1f26d26a9c6b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:55.955106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 576/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:18:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:55.960881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937cfc-b21c-4e42-8fa8-252302bffd43 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:56.057352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77937cfc-b21c-4e42-8fa8-252302bffd43 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:56.089796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82261d-a09a-4840-83de-ff23ff600b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:56.090397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82261d-a09a-4840-83de-ff23ff600b7d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:56.090669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82261d-a09a-4840-83de-ff23ff600b7d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:56.090935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82261d-a09a-4840-83de-ff23ff600b7d None admin] PATH_INFO: `/v3/users/a84fa877cab7409b968ab1f7d83e1886` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:56.092212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82261d-a09a-4840-83de-ff23ff600b7d None admin] RBAC: Authorizing `identity:get_user(user_id=a84fa877cab7409b968ab1f7d83e1886)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:56.093676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82261d-a09a-4840-83de-ff23ff600b7d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:56.095215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 582/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:18:55 2022] GET /identity/v3/users/a84fa877cab7409b968ab1f7d83e1886 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:56.101290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7276d1-5cf5-4b4d-9675-1f26d26a9c6b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:56.198395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7276d1-5cf5-4b4d-9675-1f26d26a9c6b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:56.231160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa35511-cbad-4fe2-b511-f761f3ab044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:56.231768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa35511-cbad-4fe2-b511-f761f3ab044b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:56.232045 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa35511-cbad-4fe2-b511-f761f3ab044b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:56.232317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa35511-cbad-4fe2-b511-f761f3ab044b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:56.250947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:56.299552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa35511-cbad-4fe2-b511-f761f3ab044b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:56.301034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa35511-cbad-4fe2-b511-f761f3ab044b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:56.396929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 577/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:18:56.402776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82261d-a09a-4840-83de-ff23ff600b7d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:56.412041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:56.500732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82261d-a09a-4840-83de-ff23ff600b7d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:56.532670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c183fd4-44c8-420b-b42f-0d29c3f54e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:56.533275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c183fd4-44c8-420b-b42f-0d29c3f54e39 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:56.533547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c183fd4-44c8-420b-b42f-0d29c3f54e39 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:56.533819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c183fd4-44c8-420b-b42f-0d29c3f54e39 None admin] PATH_INFO: `/v3/projects/6ae4dbb2b8aa473c9833ee83e7a96aef` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:56.534900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c183fd4-44c8-420b-b42f-0d29c3f54e39 None admin] RBAC: Authorizing `identity:get_project(project_id=6ae4dbb2b8aa473c9833ee83e7a96aef)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:56.536318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c183fd4-44c8-420b-b42f-0d29c3f54e39 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:56.537935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 583/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:18:56 2022] GET /identity/v3/projects/6ae4dbb2b8aa473c9833ee83e7a96aef => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:56.544130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa35511-cbad-4fe2-b511-f761f3ab044b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:56.591306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:56.641086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa35511-cbad-4fe2-b511-f761f3ab044b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:56.675048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a27a6-1c03-4756-98fb-319171f40adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:56.675642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a27a6-1c03-4756-98fb-319171f40adf None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:56.675922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a27a6-1c03-4756-98fb-319171f40adf None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:56.676194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a27a6-1c03-4756-98fb-319171f40adf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:56.676947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a27a6-1c03-4756-98fb-319171f40adf None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=6ae4dbb2b8aa473c9833ee83e7a96aef, user.id=a84fa877cab7409b968ab1f7d83e1886) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:56.677257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a27a6-1c03-4756-98fb-319171f40adf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:56.678683 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a27a6-1c03-4756-98fb-319171f40adf None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:56.684295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 578/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:18:56 2022] GET /identity/v3/role_assignments?user.id=a84fa877cab7409b968ab1f7d83e1886&scope.project.id=6ae4dbb2b8aa473c9833ee83e7a96aef&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:18:58.582711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604812de-4e75-458e-b36a-9c9a99ea6c29 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:58.583223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604812de-4e75-458e-b36a-9c9a99ea6c29 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:58.583339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604812de-4e75-458e-b36a-9c9a99ea6c29 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:58.584626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 584/1162] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:18:58.590030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142e26f-2b0e-4228-9554-fe6476c2b08d None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:58.590425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142e26f-2b0e-4228-9554-fe6476c2b08d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:58.590574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142e26f-2b0e-4228-9554-fe6476c2b08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:58.609421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-c142e26f-2b0e-4228-9554-fe6476c2b08d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:18:58.610605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-c142e26f-2b0e-4228-9554-fe6476c2b08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:18:58.675136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 579/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:18:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:18:58.940401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604812de-4e75-458e-b36a-9c9a99ea6c29 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:59.016893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:59.042885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604812de-4e75-458e-b36a-9c9a99ea6c29 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:59.076283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c957739-20d7-4718-b27a-72db1f5d7a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:59.077057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957739-20d7-4718-b27a-72db1f5d7a7b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:59.077332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957739-20d7-4718-b27a-72db1f5d7a7b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:59.077599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957739-20d7-4718-b27a-72db1f5d7a7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:59.142166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c957739-20d7-4718-b27a-72db1f5d7a7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:59.143592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c957739-20d7-4718-b27a-72db1f5d7a7b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:59.212913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:59.237914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 585/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:18:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:18:59.243837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142e26f-2b0e-4228-9554-fe6476c2b08d None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:59.287007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:18:59.343077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142e26f-2b0e-4228-9554-fe6476c2b08d None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:59.381513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f76fb5-e595-4e45-806d-44981b19c70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:59.382129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f76fb5-e595-4e45-806d-44981b19c70a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:59.382402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f76fb5-e595-4e45-806d-44981b19c70a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:59.382669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f76fb5-e595-4e45-806d-44981b19c70a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:59.389357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f76fb5-e595-4e45-806d-44981b19c70a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:59.390779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f76fb5-e595-4e45-806d-44981b19c70a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-b7f76fb5-e595-4e45-806d-44981b19c70a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:18:59.401732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:18:59.402910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:18:59.403756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 580/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:18:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:18:59.409423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c957739-20d7-4718-b27a-72db1f5d7a7b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:18:59.505725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c957739-20d7-4718-b27a-72db1f5d7a7b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:18:59.538324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9a25f-0062-4509-8f29-403e407c5384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:18:59.538913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9a25f-0062-4509-8f29-403e407c5384 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:18:59.539190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9a25f-0062-4509-8f29-403e407c5384 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:18:59.539463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9a25f-0062-4509-8f29-403e407c5384 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:18:59.540093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9a25f-0062-4509-8f29-403e407c5384 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:18:59.540392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9a25f-0062-4509-8f29-403e407c5384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:18:59.541786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9a25f-0062-4509-8f29-403e407c5384 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:18:59.549200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 586/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 1 00:18:59 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:01.418669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a01ca-f39c-46f6-b9d9-5197271a4db9 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:01.419216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a01ca-f39c-46f6-b9d9-5197271a4db9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:01.419216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a01ca-f39c-46f6-b9d9-5197271a4db9 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:01.421112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 581/1167] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:19:01.426455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a852b-1373-4030-a774-78d67d85a718 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:01.426859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a852b-1373-4030-a774-78d67d85a718 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:01.426995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a852b-1373-4030-a774-78d67d85a718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:01.445627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-445a852b-1373-4030-a774-78d67d85a718 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:19:01.446795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-445a852b-1373-4030-a774-78d67d85a718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:19:01.475185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:01.511502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 587/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:19:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:19:01.780486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a01ca-f39c-46f6-b9d9-5197271a4db9 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:01.880071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a01ca-f39c-46f6-b9d9-5197271a4db9 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:01.881585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:01.914493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926b1a3-3515-49e6-ab42-71f3bb99652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:01.915087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926b1a3-3515-49e6-ab42-71f3bb99652f None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:01.915363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926b1a3-3515-49e6-ab42-71f3bb99652f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:01.915634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926b1a3-3515-49e6-ab42-71f3bb99652f None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:01.916300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926b1a3-3515-49e6-ab42-71f3bb99652f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:01.917699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926b1a3-3515-49e6-ab42-71f3bb99652f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:01.933567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 582/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Thu Sep 1 00:19:01 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:19:03.833520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998a6c-99b4-462e-b4ca-4caf92eb5b9e None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:03.834317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998a6c-99b4-462e-b4ca-4caf92eb5b9e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:03.834474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5998a6c-99b4-462e-b4ca-4caf92eb5b9e None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:03.835889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 588/1170] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:19:03.841077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38941851-0cd3-457c-bc2b-54d877118577 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:03.841495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38941851-0cd3-457c-bc2b-54d877118577 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:03.842166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38941851-0cd3-457c-bc2b-54d877118577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:03.860758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-38941851-0cd3-457c-bc2b-54d877118577 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:19:03.861935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-38941851-0cd3-457c-bc2b-54d877118577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:19:03.925590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 583/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:19:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:19:04.194515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998a6c-99b4-462e-b4ca-4caf92eb5b9e None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:04.295257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5998a6c-99b4-462e-b4ca-4caf92eb5b9e None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:04.327710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007ccea-1fd6-463a-b496-e6f306b9da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:04.328330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007ccea-1fd6-463a-b496-e6f306b9da1d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:04.328630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007ccea-1fd6-463a-b496-e6f306b9da1d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:04.328901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007ccea-1fd6-463a-b496-e6f306b9da1d None admin] PATH_INFO: `/v3/domains/1f4ef45b4da34970900d0fb2b6045dd6` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:04.338970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007ccea-1fd6-463a-b496-e6f306b9da1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f4ef45b4da34970900d0fb2b6045dd6)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:04.340416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007ccea-1fd6-463a-b496-e6f306b9da1d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:04.342029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 589/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 1 00:19:04 2022] GET /identity/v3/domains/1f4ef45b4da34970900d0fb2b6045dd6 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:04.347927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38941851-0cd3-457c-bc2b-54d877118577 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:04.437074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:04.447164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38941851-0cd3-457c-bc2b-54d877118577 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:04.479629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c2ab0-899c-4191-b6c7-c152ad56c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:04.480231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c2ab0-899c-4191-b6c7-c152ad56c3be None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:04.480528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c2ab0-899c-4191-b6c7-c152ad56c3be None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:04.480808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c2ab0-899c-4191-b6c7-c152ad56c3be None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:04.481450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c2ab0-899c-4191-b6c7-c152ad56c3be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:04.483421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c2ab0-899c-4191-b6c7-c152ad56c3be None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:04.508848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 584/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:19:04 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:19:06.422297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d612ed-e836-408b-a761-ec1f1b0551a3 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:06.422821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d612ed-e836-408b-a761-ec1f1b0551a3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:06.422821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d612ed-e836-408b-a761-ec1f1b0551a3 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:06.424163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 590/1174] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:19:06.429269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d99da9-c92c-49b0-be5a-af14011cef44 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:06.429664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d99da9-c92c-49b0-be5a-af14011cef44 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:06.429815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d99da9-c92c-49b0-be5a-af14011cef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:06.449138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-a8d99da9-c92c-49b0-be5a-af14011cef44 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:19:06.450320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a8d99da9-c92c-49b0-be5a-af14011cef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:19:06.515693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 585/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:19:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:19:06.784601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d612ed-e836-408b-a761-ec1f1b0551a3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:06.813844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:06.885108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d612ed-e836-408b-a761-ec1f1b0551a3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:06.917823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c7d5b-cd5f-4df2-8a3d-66e534c9f945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:06.918436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c7d5b-cd5f-4df2-8a3d-66e534c9f945 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:06.918714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c7d5b-cd5f-4df2-8a3d-66e534c9f945 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:06.918987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c7d5b-cd5f-4df2-8a3d-66e534c9f945 None admin] PATH_INFO: `/v3/domains/1f4ef45b4da34970900d0fb2b6045dd6` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:06.920800 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c7d5b-cd5f-4df2-8a3d-66e534c9f945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f4ef45b4da34970900d0fb2b6045dd6)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:06.922307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c7d5b-cd5f-4df2-8a3d-66e534c9f945 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:06.923852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 591/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 1 00:19:06 2022] GET /identity/v3/domains/1f4ef45b4da34970900d0fb2b6045dd6 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:06.929536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d99da9-c92c-49b0-be5a-af14011cef44 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:07.000334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:07.024715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d99da9-c92c-49b0-be5a-af14011cef44 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:07.060939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776674fb-5bab-4e9c-bbbb-cd08f28fd008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:07.061534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776674fb-5bab-4e9c-bbbb-cd08f28fd008 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:07.061806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776674fb-5bab-4e9c-bbbb-cd08f28fd008 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:07.062071 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776674fb-5bab-4e9c-bbbb-cd08f28fd008 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:07.062703 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776674fb-5bab-4e9c-bbbb-cd08f28fd008 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:07.064131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776674fb-5bab-4e9c-bbbb-cd08f28fd008 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:07.084506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 586/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:19:06 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:19:09.030145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bde12f-027b-4401-b65c-291c9d101bb4 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:09.030678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bde12f-027b-4401-b65c-291c9d101bb4 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:09.030678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bde12f-027b-4401-b65c-291c9d101bb4 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:09.032018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 592/1178] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:19:09.037025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ced25-efdf-4f58-a4b9-71a7b5960efb None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:09.037430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ced25-efdf-4f58-a4b9-71a7b5960efb None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:09.037567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ced25-efdf-4f58-a4b9-71a7b5960efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:09.055083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-742ced25-efdf-4f58-a4b9-71a7b5960efb None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:19:09.056317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-742ced25-efdf-4f58-a4b9-71a7b5960efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:19:09.121386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 587/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:19:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:19:09.386825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bde12f-027b-4401-b65c-291c9d101bb4 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:09.479995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:09.487490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bde12f-027b-4401-b65c-291c9d101bb4 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:09.520475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd304322-6690-40b6-9aba-8818179e8b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:09.521113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd304322-6690-40b6-9aba-8818179e8b5e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:09.521393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd304322-6690-40b6-9aba-8818179e8b5e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:09.521665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd304322-6690-40b6-9aba-8818179e8b5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:09.532029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd304322-6690-40b6-9aba-8818179e8b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-bd304322-6690-40b6-9aba-8818179e8b5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:19:09.537004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:19:09.537930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:19:09.538798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 593/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:19:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:19:09.544015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ced25-efdf-4f58-a4b9-71a7b5960efb None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:09.600774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:09.639845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ced25-efdf-4f58-a4b9-71a7b5960efb None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:09.673259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5da620-b3f4-4544-b22a-4e7845281062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:09.673850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5da620-b3f4-4544-b22a-4e7845281062 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:09.674124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5da620-b3f4-4544-b22a-4e7845281062 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:09.674392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5da620-b3f4-4544-b22a-4e7845281062 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:09.675043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5da620-b3f4-4544-b22a-4e7845281062 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:19:09.675340 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5da620-b3f4-4544-b22a-4e7845281062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:09.676726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5da620-b3f4-4544-b22a-4e7845281062 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:09.684432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 588/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:19:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:09.690525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd304322-6690-40b6-9aba-8818179e8b5e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:09.790049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd304322-6690-40b6-9aba-8818179e8b5e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:09.823257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d71e59-9a1e-488c-8017-02575351d5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:09.823872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d71e59-9a1e-488c-8017-02575351d5d8 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:09.824150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d71e59-9a1e-488c-8017-02575351d5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:09.824422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d71e59-9a1e-488c-8017-02575351d5d8 None admin] PATH_INFO: `/v3/users/170ca5671ff54a95896e90e9d1a13f14` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:09.837653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d71e59-9a1e-488c-8017-02575351d5d8 None admin] RBAC: Authorizing `identity:get_user(user_id=170ca5671ff54a95896e90e9d1a13f14)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:09.839099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d71e59-9a1e-488c-8017-02575351d5d8 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:09.840704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 594/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:19:09 2022] GET /identity/v3/users/170ca5671ff54a95896e90e9d1a13f14 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:09.846819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5da620-b3f4-4544-b22a-4e7845281062 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:09.943183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5da620-b3f4-4544-b22a-4e7845281062 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:09.964534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:09.976342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c66532-4dc0-46ab-9594-4a855e006dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:09.976953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c66532-4dc0-46ab-9594-4a855e006dcd None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:09.977230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c66532-4dc0-46ab-9594-4a855e006dcd None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:09.977494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c66532-4dc0-46ab-9594-4a855e006dcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:10.042012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c66532-4dc0-46ab-9594-4a855e006dcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:10.043453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c66532-4dc0-46ab-9594-4a855e006dcd None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:10.137296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 589/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:19:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:19:10.143420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d71e59-9a1e-488c-8017-02575351d5d8 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:10.159807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:10.241933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d71e59-9a1e-488c-8017-02575351d5d8 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:10.276186 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81be052-f4a1-4225-90be-4e7a0eefe55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:10.276841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81be052-f4a1-4225-90be-4e7a0eefe55a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:10.277122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81be052-f4a1-4225-90be-4e7a0eefe55a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:10.277394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81be052-f4a1-4225-90be-4e7a0eefe55a None admin] PATH_INFO: `/v3/projects/4dbd1083270042039d1f4fc3b761df69` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:10.278511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81be052-f4a1-4225-90be-4e7a0eefe55a None admin] RBAC: Authorizing `identity:get_project(project_id=4dbd1083270042039d1f4fc3b761df69)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:10.280470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81be052-f4a1-4225-90be-4e7a0eefe55a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:10.282121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 595/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:19:10 2022] GET /identity/v3/projects/4dbd1083270042039d1f4fc3b761df69 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:10.288128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c66532-4dc0-46ab-9594-4a855e006dcd None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:10.311792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:10.384040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c66532-4dc0-46ab-9594-4a855e006dcd None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:10.415994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070f43b-7712-4bdc-8991-12b620593fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:10.416605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f43b-7712-4bdc-8991-12b620593fe2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:10.416880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f43b-7712-4bdc-8991-12b620593fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:10.417144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070f43b-7712-4bdc-8991-12b620593fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:10.417843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070f43b-7712-4bdc-8991-12b620593fe2 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=4dbd1083270042039d1f4fc3b761df69, user.id=170ca5671ff54a95896e90e9d1a13f14) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:19:10.418142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070f43b-7712-4bdc-8991-12b620593fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:10.420051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070f43b-7712-4bdc-8991-12b620593fe2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:10.425204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 590/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:19:10 2022] GET /identity/v3/role_assignments?user.id=170ca5671ff54a95896e90e9d1a13f14&scope.project.id=4dbd1083270042039d1f4fc3b761df69&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:12.296820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d41fd0-d4a1-452b-8437-67f074f9b109 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:12.297432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d41fd0-d4a1-452b-8437-67f074f9b109 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:12.297432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d41fd0-d4a1-452b-8437-67f074f9b109 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:12.298764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 596/1186] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:19:12.303739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbfd4a-4450-42dc-a0b9-18aef322fece None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:12.304188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbfd4a-4450-42dc-a0b9-18aef322fece None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:12.304296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbfd4a-4450-42dc-a0b9-18aef322fece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:12.323727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-15bbfd4a-4450-42dc-a0b9-18aef322fece None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:19:12.324972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-15bbfd4a-4450-42dc-a0b9-18aef322fece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:19:12.390940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 591/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:19:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:19:12.656127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d41fd0-d4a1-452b-8437-67f074f9b109 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:12.737690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:12.756974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d41fd0-d4a1-452b-8437-67f074f9b109 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:12.789516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b3ceb-5bba-4f51-8813-eb763ebd589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:12.790128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b3ceb-5bba-4f51-8813-eb763ebd589d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:12.790405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b3ceb-5bba-4f51-8813-eb763ebd589d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:12.790674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b3ceb-5bba-4f51-8813-eb763ebd589d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:12.801026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b3ceb-5bba-4f51-8813-eb763ebd589d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-b86b3ceb-5bba-4f51-8813-eb763ebd589d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:19:12.805812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:19:12.806979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:19:12.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 597/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:19:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:19:12.813495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbfd4a-4450-42dc-a0b9-18aef322fece None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:12.830445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:12.912726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbfd4a-4450-42dc-a0b9-18aef322fece None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:12.945853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27720d77-1035-47e3-a7f7-2dd5f13e13ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:12.946448 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27720d77-1035-47e3-a7f7-2dd5f13e13ce None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:12.946729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27720d77-1035-47e3-a7f7-2dd5f13e13ce None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:12.947003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27720d77-1035-47e3-a7f7-2dd5f13e13ce None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:12.947658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27720d77-1035-47e3-a7f7-2dd5f13e13ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:19:12.947960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27720d77-1035-47e3-a7f7-2dd5f13e13ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:12.949364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27720d77-1035-47e3-a7f7-2dd5f13e13ce None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:12.957185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 592/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:19:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:12.962985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b3ceb-5bba-4f51-8813-eb763ebd589d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:13.061263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b3ceb-5bba-4f51-8813-eb763ebd589d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:13.094540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cfbe8-5bff-40c8-9700-819d6491f4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:13.095144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cfbe8-5bff-40c8-9700-819d6491f4d9 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:13.095417 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cfbe8-5bff-40c8-9700-819d6491f4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:13.095685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cfbe8-5bff-40c8-9700-819d6491f4d9 None admin] PATH_INFO: `/v3/users/170ca5671ff54a95896e90e9d1a13f14` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:13.096807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cfbe8-5bff-40c8-9700-819d6491f4d9 None admin] RBAC: Authorizing `identity:get_user(user_id=170ca5671ff54a95896e90e9d1a13f14)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:13.098482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cfbe8-5bff-40c8-9700-819d6491f4d9 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:13.100015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 598/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:19:12 2022] GET /identity/v3/users/170ca5671ff54a95896e90e9d1a13f14 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:13.105687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27720d77-1035-47e3-a7f7-2dd5f13e13ce None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:13.182815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:13.204478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27720d77-1035-47e3-a7f7-2dd5f13e13ce None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:13.238447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0e4ef-abf4-4605-82a4-a62b0403e94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:13.239063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e4ef-abf4-4605-82a4-a62b0403e94a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:13.239337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e4ef-abf4-4605-82a4-a62b0403e94a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:13.239606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0e4ef-abf4-4605-82a4-a62b0403e94a None admin] PATH_INFO: `/v3/projects/4dbd1083270042039d1f4fc3b761df69` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:13.240751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0e4ef-abf4-4605-82a4-a62b0403e94a None admin] RBAC: Authorizing `identity:get_project(project_id=4dbd1083270042039d1f4fc3b761df69)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:13.242182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0e4ef-abf4-4605-82a4-a62b0403e94a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:13.243799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 593/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:19:13 2022] GET /identity/v3/projects/4dbd1083270042039d1f4fc3b761df69 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:13.250421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cfbe8-5bff-40c8-9700-819d6491f4d9 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:13.262940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:13.347173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cfbe8-5bff-40c8-9700-819d6491f4d9 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:13.379991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a81f1-7422-400f-8cfb-89a9f61a2b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:13.380654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a81f1-7422-400f-8cfb-89a9f61a2b48 None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:13.380927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a81f1-7422-400f-8cfb-89a9f61a2b48 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:13.381194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a81f1-7422-400f-8cfb-89a9f61a2b48 None admin] PATH_INFO: `/v3/projects/4dbd1083270042039d1f4fc3b761df69/users/170ca5671ff54a95896e90e9d1a13f14/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:13.383029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a81f1-7422-400f-8cfb-89a9f61a2b48 None admin] RBAC: Authorizing `identity:create_grant(project_id=4dbd1083270042039d1f4fc3b761df69, user_id=170ca5671ff54a95896e90e9d1a13f14, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:13.384449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a81f1-7422-400f-8cfb-89a9f61a2b48 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:13.397101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 599/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:19:13 2022] PUT /identity/v3/projects/4dbd1083270042039d1f4fc3b761df69/users/170ca5671ff54a95896e90e9d1a13f14/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:19:15.261037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb91a96-81fc-4d82-b0ef-0558870be9c6 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:15.261481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb91a96-81fc-4d82-b0ef-0558870be9c6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:15.261584 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb91a96-81fc-4d82-b0ef-0558870be9c6 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:15.262937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 594/1193] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:19:15.268221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a69b5-db8d-4561-858a-22f7af924b8a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:15.268630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a69b5-db8d-4561-858a-22f7af924b8a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:15.268815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a69b5-db8d-4561-858a-22f7af924b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:15.289000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d14a69b5-db8d-4561-858a-22f7af924b8a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:19:15.290165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d14a69b5-db8d-4561-858a-22f7af924b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:19:15.350782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:15.356085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 600/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:19:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:19:15.621438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb91a96-81fc-4d82-b0ef-0558870be9c6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:15.720864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb91a96-81fc-4d82-b0ef-0558870be9c6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:15.754951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4040b6-1b6b-4509-8d16-3a46014eb432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:15.755557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4040b6-1b6b-4509-8d16-3a46014eb432 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:15.755831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4040b6-1b6b-4509-8d16-3a46014eb432 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:15.756098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4040b6-1b6b-4509-8d16-3a46014eb432 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:15.765781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4040b6-1b6b-4509-8d16-3a46014eb432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-aa4040b6-1b6b-4509-8d16-3a46014eb432 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:19:15.771780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:19:15.772933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:19:15.773992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 595/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:19:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:19:15.779033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a69b5-db8d-4561-858a-22f7af924b8a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:15.874877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a69b5-db8d-4561-858a-22f7af924b8a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:15.907630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e3f1a-4c20-4383-9824-63bb606f4d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:15.908215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3f1a-4c20-4383-9824-63bb606f4d1b None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:15.908490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3f1a-4c20-4383-9824-63bb606f4d1b None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:15.908788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e3f1a-4c20-4383-9824-63bb606f4d1b None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:15.909437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e3f1a-4c20-4383-9824-63bb606f4d1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:19:15.909732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e3f1a-4c20-4383-9824-63bb606f4d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:15.911091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e3f1a-4c20-4383-9824-63bb606f4d1b None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:15.918830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 601/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:19:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:15.926170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4040b6-1b6b-4509-8d16-3a46014eb432 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:15.936038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:16.025428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4040b6-1b6b-4509-8d16-3a46014eb432 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:16.057720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39415ec-ed4f-468f-b194-9c11a4c1be50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:16.058332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39415ec-ed4f-468f-b194-9c11a4c1be50 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:16.058609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39415ec-ed4f-468f-b194-9c11a4c1be50 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:16.058878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39415ec-ed4f-468f-b194-9c11a4c1be50 None admin] PATH_INFO: `/v3/users/170ca5671ff54a95896e90e9d1a13f14` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:16.060621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39415ec-ed4f-468f-b194-9c11a4c1be50 None admin] RBAC: Authorizing `identity:get_user(user_id=170ca5671ff54a95896e90e9d1a13f14)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:16.062053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39415ec-ed4f-468f-b194-9c11a4c1be50 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:16.063609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 596/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 1 00:19:15 2022] GET /identity/v3/users/170ca5671ff54a95896e90e9d1a13f14 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:16.069630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e3f1a-4c20-4383-9824-63bb606f4d1b None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:16.130606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:16.168542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e3f1a-4c20-4383-9824-63bb606f4d1b None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:16.201756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309f146-dfa6-4058-beb0-43233a176785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:16.202342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309f146-dfa6-4058-beb0-43233a176785 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:16.202619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309f146-dfa6-4058-beb0-43233a176785 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:16.202895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309f146-dfa6-4058-beb0-43233a176785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:16.269514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309f146-dfa6-4058-beb0-43233a176785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:16.270957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309f146-dfa6-4058-beb0-43233a176785 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:16.328616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:16.367044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 602/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:19:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:19:16.373154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39415ec-ed4f-468f-b194-9c11a4c1be50 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:16.445417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:16.470362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39415ec-ed4f-468f-b194-9c11a4c1be50 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:16.503079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b0f1-4b1b-4062-8186-b041e69217a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:16.503691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b0f1-4b1b-4062-8186-b041e69217a7 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:16.503968 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b0f1-4b1b-4062-8186-b041e69217a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:16.504237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b0f1-4b1b-4062-8186-b041e69217a7 None admin] PATH_INFO: `/v3/projects/4dbd1083270042039d1f4fc3b761df69` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:16.505369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b0f1-4b1b-4062-8186-b041e69217a7 None admin] RBAC: Authorizing `identity:get_project(project_id=4dbd1083270042039d1f4fc3b761df69)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:16.506796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b0f1-4b1b-4062-8186-b041e69217a7 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:16.508399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 597/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 1 00:19:16 2022] GET /identity/v3/projects/4dbd1083270042039d1f4fc3b761df69 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:16.515195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309f146-dfa6-4058-beb0-43233a176785 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:19:16.611450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309f146-dfa6-4058-beb0-43233a176785 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:19:16.644197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377602-5da8-456e-9fe8-a54a880ffc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:19:16.644800 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377602-5da8-456e-9fe8-a54a880ffc06 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:19:16.645073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377602-5da8-456e-9fe8-a54a880ffc06 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:19:16.645335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377602-5da8-456e-9fe8-a54a880ffc06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:19:16.646039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377602-5da8-456e-9fe8-a54a880ffc06 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=4dbd1083270042039d1f4fc3b761df69, user.id=170ca5671ff54a95896e90e9d1a13f14) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:19:16.646336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377602-5da8-456e-9fe8-a54a880ffc06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:19:16.647735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377602-5da8-456e-9fe8-a54a880ffc06 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:19:16.653313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 603/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:19:16 2022] GET /identity/v3/role_assignments?user.id=170ca5671ff54a95896e90e9d1a13f14&scope.project.id=4dbd1083270042039d1f4fc3b761df69&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:19:26.499478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=29 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:19:26.650773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=87 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:11.661752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c17e55-a6cc-4ac8-a74d-b78641c8ad3a None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:11.662354 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c17e55-a6cc-4ac8-a74d-b78641c8ad3a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:11.662354 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c17e55-a6cc-4ac8-a74d-b78641c8ad3a None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:11.664069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 598/1201] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:11.669798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c4b47-a346-4593-98a9-9d96555da094 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:11.670264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c4b47-a346-4593-98a9-9d96555da094 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:11.670351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c4b47-a346-4593-98a9-9d96555da094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:11.694193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-710c4b47-a346-4593-98a9-9d96555da094 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:11.695475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-710c4b47-a346-4593-98a9-9d96555da094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:11.779997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 604/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:12.077044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c17e55-a6cc-4ac8-a74d-b78641c8ad3a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:12.223800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c17e55-a6cc-4ac8-a74d-b78641c8ad3a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:12.285734 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc2b56-ea0c-4a2c-bf1a-967b4625a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:12.286464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc2b56-ea0c-4a2c-bf1a-967b4625a87e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:12.286746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc2b56-ea0c-4a2c-bf1a-967b4625a87e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:12.287009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc2b56-ea0c-4a2c-bf1a-967b4625a87e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:12.300527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc2b56-ea0c-4a2c-bf1a-967b4625a87e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:12.302202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc2b56-ea0c-4a2c-bf1a-967b4625a87e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-acdc2b56-ea0c-4a2c-bf1a-967b4625a87e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:12.318700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:12.325168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:12.326171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:12.326171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:12.326171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:12.326171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 599/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:12.326428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c4b47-a346-4593-98a9-9d96555da094 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:12.482243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c4b47-a346-4593-98a9-9d96555da094 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:12.517951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:12.526083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2ac1b-5873-4ad4-ad47-734fc5e678e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:12.526766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2ac1b-5873-4ad4-ad47-734fc5e678e5 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:12.527134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2ac1b-5873-4ad4-ad47-734fc5e678e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:12.527405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2ac1b-5873-4ad4-ad47-734fc5e678e5 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:12.528063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2ac1b-5873-4ad4-ad47-734fc5e678e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:12.528388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2ac1b-5873-4ad4-ad47-734fc5e678e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:12.529968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2ac1b-5873-4ad4-ad47-734fc5e678e5 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:12.542401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 605/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:12.548072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc2b56-ea0c-4a2c-bf1a-967b4625a87e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:12.619440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:12.677846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc2b56-ea0c-4a2c-bf1a-967b4625a87e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:12.718780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147834ee-9329-4bf1-87af-9a76aecf3edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:12.719397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147834ee-9329-4bf1-87af-9a76aecf3edf None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:12.719664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147834ee-9329-4bf1-87af-9a76aecf3edf None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:12.719940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147834ee-9329-4bf1-87af-9a76aecf3edf None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:12.720791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147834ee-9329-4bf1-87af-9a76aecf3edf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:12.722497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147834ee-9329-4bf1-87af-9a76aecf3edf None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:12.747735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 600/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 1 00:20:12 2022] POST /identity/v3/users => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:20:14.816080 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5126cf-c919-411e-98b7-70dab1c05aab None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:14.816781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5126cf-c919-411e-98b7-70dab1c05aab None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:14.816781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5126cf-c919-411e-98b7-70dab1c05aab None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:14.818157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 606/1206] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:20:14.823866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa44e1-9cac-4203-bd84-78767ab26bfd None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:14.824363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa44e1-9cac-4203-bd84-78767ab26bfd None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:14.824464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa44e1-9cac-4203-bd84-78767ab26bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:14.843897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-79fa44e1-9cac-4203-bd84-78767ab26bfd None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:14.845133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-79fa44e1-9cac-4203-bd84-78767ab26bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:14.924370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 601/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:15.202954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5126cf-c919-411e-98b7-70dab1c05aab None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:15.331997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5126cf-c919-411e-98b7-70dab1c05aab None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:15.379364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9548b-b6ae-46b3-af05-6a86b33c74ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:15.380015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9548b-b6ae-46b3-af05-6a86b33c74ea None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:15.380328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9548b-b6ae-46b3-af05-6a86b33c74ea None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:15.380735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9548b-b6ae-46b3-af05-6a86b33c74ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:15.393353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9548b-b6ae-46b3-af05-6a86b33c74ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-a7d9548b-b6ae-46b3-af05-6a86b33c74ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:20:15.398891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:15.400115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:15.401537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 607/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:20:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:15.405714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa44e1-9cac-4203-bd84-78767ab26bfd None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:15.449701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:15.523984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa44e1-9cac-4203-bd84-78767ab26bfd None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:15.562139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f2e44-e1df-4b1a-8801-7da5721ac2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:15.562748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f2e44-e1df-4b1a-8801-7da5721ac2d9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:15.563028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f2e44-e1df-4b1a-8801-7da5721ac2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:15.563296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f2e44-e1df-4b1a-8801-7da5721ac2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:15.563964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f2e44-e1df-4b1a-8801-7da5721ac2d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:15.564270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f2e44-e1df-4b1a-8801-7da5721ac2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:15.565682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f2e44-e1df-4b1a-8801-7da5721ac2d9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:15.579624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 602/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:20:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:15.580996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9548b-b6ae-46b3-af05-6a86b33c74ea None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:15.639529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:15.703883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9548b-b6ae-46b3-af05-6a86b33c74ea None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:15.741706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331d2cc-6841-4ab3-acb7-7aef77737fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:15.742308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d2cc-6841-4ab3-acb7-7aef77737fa6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:15.742609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d2cc-6841-4ab3-acb7-7aef77737fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:15.742852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d2cc-6841-4ab3-acb7-7aef77737fa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:15.755981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d2cc-6841-4ab3-acb7-7aef77737fa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:15.757533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d2cc-6841-4ab3-acb7-7aef77737fa6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-6331d2cc-6841-4ab3-acb7-7aef77737fa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:15.773712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:15.774816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:15.775964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:15.775964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:15.775964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:15.775964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 608/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:20:15.781479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f2e44-e1df-4b1a-8801-7da5721ac2d9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:15.896515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:15.899601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f2e44-e1df-4b1a-8801-7da5721ac2d9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:15.938091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4d80e-f896-46e9-abaa-9d0ed85b9a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:15.938864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4d80e-f896-46e9-abaa-9d0ed85b9a0d None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:15.939457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4d80e-f896-46e9-abaa-9d0ed85b9a0d None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:15.939738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4d80e-f896-46e9-abaa-9d0ed85b9a0d None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:15.940553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4d80e-f896-46e9-abaa-9d0ed85b9a0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:15.940868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4d80e-f896-46e9-abaa-9d0ed85b9a0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:15.942510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4d80e-f896-46e9-abaa-9d0ed85b9a0d None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:15.961556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 603/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:15.963106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331d2cc-6841-4ab3-acb7-7aef77737fa6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:16.091219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331d2cc-6841-4ab3-acb7-7aef77737fa6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:16.093427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:16.134875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f12ef-16ff-476e-9ba4-ecfc8dc317d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:16.135602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f12ef-16ff-476e-9ba4-ecfc8dc317d7 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:16.135877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f12ef-16ff-476e-9ba4-ecfc8dc317d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:16.136220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f12ef-16ff-476e-9ba4-ecfc8dc317d7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:16.144735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f12ef-16ff-476e-9ba4-ecfc8dc317d7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:16.145904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f12ef-16ff-476e-9ba4-ecfc8dc317d7 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-625f12ef-16ff-476e-9ba4-ecfc8dc317d7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:20:16.160214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:16.161419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:16.162387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 609/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:20:16.168723 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4d80e-f896-46e9-abaa-9d0ed85b9a0d None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:16.288739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4d80e-f896-46e9-abaa-9d0ed85b9a0d None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:16.333600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72807d52-761b-48d4-8e15-2f91dad831f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:16.334594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72807d52-761b-48d4-8e15-2f91dad831f8 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:16.335088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72807d52-761b-48d4-8e15-2f91dad831f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:16.335364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72807d52-761b-48d4-8e15-2f91dad831f8 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:16.336166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72807d52-761b-48d4-8e15-2f91dad831f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:16.336472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72807d52-761b-48d4-8e15-2f91dad831f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:16.338499 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72807d52-761b-48d4-8e15-2f91dad831f8 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:16.358348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 604/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 00:20:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:16.363877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f12ef-16ff-476e-9ba4-ecfc8dc317d7 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:16.493715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f12ef-16ff-476e-9ba4-ecfc8dc317d7 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:16.538530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c79bb-78dd-486f-83e3-a0f55843c26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:16.539346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c79bb-78dd-486f-83e3-a0f55843c26a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:16.539668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c79bb-78dd-486f-83e3-a0f55843c26a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:16.540096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c79bb-78dd-486f-83e3-a0f55843c26a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:16.574092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:16.624792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c79bb-78dd-486f-83e3-a0f55843c26a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:16.626344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c79bb-78dd-486f-83e3-a0f55843c26a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:16.746086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 610/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:20:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:20:16.747285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72807d52-761b-48d4-8e15-2f91dad831f8 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:16.767618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:16.873815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72807d52-761b-48d4-8e15-2f91dad831f8 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:16.909989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80862ec4-8f1a-4cf9-8f82-73d247aa5cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:16.910782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80862ec4-8f1a-4cf9-8f82-73d247aa5cd5 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:16.910894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80862ec4-8f1a-4cf9-8f82-73d247aa5cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:16.911124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80862ec4-8f1a-4cf9-8f82-73d247aa5cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:16.921977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80862ec4-8f1a-4cf9-8f82-73d247aa5cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:16.923466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80862ec4-8f1a-4cf9-8f82-73d247aa5cd5 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-80862ec4-8f1a-4cf9-8f82-73d247aa5cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:16.940678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:16.941735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:16.942615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:16.942615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:16.942615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:16.942615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 605/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:16.948758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c79bb-78dd-486f-83e3-a0f55843c26a None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:17.018644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:17.061149 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c79bb-78dd-486f-83e3-a0f55843c26a None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:17.098827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb6a96-0770-441c-9ba1-3d39186b18c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:17.099496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb6a96-0770-441c-9ba1-3d39186b18c1 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:17.102172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb6a96-0770-441c-9ba1-3d39186b18c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:17.102172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb6a96-0770-441c-9ba1-3d39186b18c1 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:17.102172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb6a96-0770-441c-9ba1-3d39186b18c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:17.102172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb6a96-0770-441c-9ba1-3d39186b18c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:17.102497 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb6a96-0770-441c-9ba1-3d39186b18c1 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:17.120087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 611/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:20:17.121567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80862ec4-8f1a-4cf9-8f82-73d247aa5cd5 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:17.202886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:17.245730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80862ec4-8f1a-4cf9-8f82-73d247aa5cd5 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:17.285689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7903f94-55f3-40d5-8d2f-b7a30a835c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:17.286306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7903f94-55f3-40d5-8d2f-b7a30a835c7b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:17.286584 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7903f94-55f3-40d5-8d2f-b7a30a835c7b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:17.286906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7903f94-55f3-40d5-8d2f-b7a30a835c7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:17.294735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7903f94-55f3-40d5-8d2f-b7a30a835c7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:17.296218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7903f94-55f3-40d5-8d2f-b7a30a835c7b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-c7903f94-55f3-40d5-8d2f-b7a30a835c7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:20:17.308309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:17.309326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:17.310214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:20:17.310214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:20:17.310214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:20:17.310214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:20:17.310214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:20:17.310214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:20:17.310214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:17.310214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:17.314468 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 606/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:20:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:17.315451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb6a96-0770-441c-9ba1-3d39186b18c1 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:17.440268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb6a96-0770-441c-9ba1-3d39186b18c1 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:17.472769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:17.486220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb45be-2cbd-4e41-ad54-36cda8deebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:17.487182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb45be-2cbd-4e41-ad54-36cda8deebeb None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:17.487690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb45be-2cbd-4e41-ad54-36cda8deebeb None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:17.488187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb45be-2cbd-4e41-ad54-36cda8deebeb None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:17.489147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb45be-2cbd-4e41-ad54-36cda8deebeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:17.489661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb45be-2cbd-4e41-ad54-36cda8deebeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:17.491459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb45be-2cbd-4e41-ad54-36cda8deebeb None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:17.505214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 612/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:20:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:20:17.512566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7903f94-55f3-40d5-8d2f-b7a30a835c7b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:17.634384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7903f94-55f3-40d5-8d2f-b7a30a835c7b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:17.666677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:17.676884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee401a-2f9d-4cb9-81d4-7dac40ecf1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:17.677533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee401a-2f9d-4cb9-81d4-7dac40ecf1ae None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:17.677862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee401a-2f9d-4cb9-81d4-7dac40ecf1ae None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:17.678172 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee401a-2f9d-4cb9-81d4-7dac40ecf1ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:17.678972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee401a-2f9d-4cb9-81d4-7dac40ecf1ae None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=e30caaea6d104a53a8654442c2383b7e) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:17.679351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee401a-2f9d-4cb9-81d4-7dac40ecf1ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:17.681020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee401a-2f9d-4cb9-81d4-7dac40ecf1ae None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:17.697683 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 607/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:20:17 2022] GET /identity/v3/role_assignments?user.id=e30caaea6d104a53a8654442c2383b7e&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:19.640480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff98dd-acbe-46b0-9e65-c4558be78c30 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:19.641028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff98dd-acbe-46b0-9e65-c4558be78c30 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:19.641107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff98dd-acbe-46b0-9e65-c4558be78c30 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:19.642400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 613/1220] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:19.647637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37af9f-d885-470d-9c13-12c78a943b29 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:19.648059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37af9f-d885-470d-9c13-12c78a943b29 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:19.648183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37af9f-d885-470d-9c13-12c78a943b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:19.667142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-6d37af9f-d885-470d-9c13-12c78a943b29 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:19.668380 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6d37af9f-d885-470d-9c13-12c78a943b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:19.739454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 608/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:20.012078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff98dd-acbe-46b0-9e65-c4558be78c30 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:20.118545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff98dd-acbe-46b0-9e65-c4558be78c30 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:20.155543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b414f9-adde-438d-b9af-7367ac76a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:20.156175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b414f9-adde-438d-b9af-7367ac76a5fa None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:20.156452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b414f9-adde-438d-b9af-7367ac76a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:20.156753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b414f9-adde-438d-b9af-7367ac76a5fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:20.168012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b414f9-adde-438d-b9af-7367ac76a5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-b9b414f9-adde-438d-b9af-7367ac76a5fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:20:20.175001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:20.176223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:20.177343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 614/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:20:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:20.183818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37af9f-d885-470d-9c13-12c78a943b29 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:20.287708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37af9f-d885-470d-9c13-12c78a943b29 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:20.314643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:20.325857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0c0a-189a-40f5-8d02-718951f01cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:20.326466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0c0a-189a-40f5-8d02-718951f01cd2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:20.326749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0c0a-189a-40f5-8d02-718951f01cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:20.327020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0c0a-189a-40f5-8d02-718951f01cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:20.327697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e0c0a-189a-40f5-8d02-718951f01cd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:20.327990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e0c0a-189a-40f5-8d02-718951f01cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:20.329408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e0c0a-189a-40f5-8d02-718951f01cd2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:20.338548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 609/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:20:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:20.344332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b414f9-adde-438d-b9af-7367ac76a5fa None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:20.391743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:20.451317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b414f9-adde-438d-b9af-7367ac76a5fa None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:20.487717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e8fa7-b357-417e-ac20-0301cbd7c635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:20.488512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e8fa7-b357-417e-ac20-0301cbd7c635 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:20.488899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e8fa7-b357-417e-ac20-0301cbd7c635 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:20.489269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e8fa7-b357-417e-ac20-0301cbd7c635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:20.499458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e8fa7-b357-417e-ac20-0301cbd7c635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:20.500937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e8fa7-b357-417e-ac20-0301cbd7c635 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-c82e8fa7-b357-417e-ac20-0301cbd7c635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:20.515628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:20.516856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:20.517991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:20.517991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:20.517991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:20.517991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 615/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:20:20.524582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0c0a-189a-40f5-8d02-718951f01cd2 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:20.629262 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0c0a-189a-40f5-8d02-718951f01cd2 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:20.662888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131dc19-ad23-41c7-8c29-190e1f29ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:20.663519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131dc19-ad23-41c7-8c29-190e1f29ef9b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:20.663756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131dc19-ad23-41c7-8c29-190e1f29ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:20.664026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131dc19-ad23-41c7-8c29-190e1f29ef9b None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:20.664773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131dc19-ad23-41c7-8c29-190e1f29ef9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:20.665101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131dc19-ad23-41c7-8c29-190e1f29ef9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:20.666579 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131dc19-ad23-41c7-8c29-190e1f29ef9b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:20.679026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 610/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:20.685581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e8fa7-b357-417e-ac20-0301cbd7c635 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:20.785444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:20.794085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e8fa7-b357-417e-ac20-0301cbd7c635 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:20.833052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7fc4d-15cf-4e06-8003-8d651c5ac3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:20.833680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7fc4d-15cf-4e06-8003-8d651c5ac3b6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:20.833954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7fc4d-15cf-4e06-8003-8d651c5ac3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:20.834212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7fc4d-15cf-4e06-8003-8d651c5ac3b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:20.842220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7fc4d-15cf-4e06-8003-8d651c5ac3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:20.843661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7fc4d-15cf-4e06-8003-8d651c5ac3b6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-06c7fc4d-15cf-4e06-8003-8d651c5ac3b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:20:20.857932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:20.859369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:20.860376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 616/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:20.865000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131dc19-ad23-41c7-8c29-190e1f29ef9b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:20.917523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:20.980415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131dc19-ad23-41c7-8c29-190e1f29ef9b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:21.017254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96240e-6b52-48ee-bf95-da0c2b4ee5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:21.017951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96240e-6b52-48ee-bf95-da0c2b4ee5b4 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:21.018229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96240e-6b52-48ee-bf95-da0c2b4ee5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:21.018493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96240e-6b52-48ee-bf95-da0c2b4ee5b4 None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:21.019292 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96240e-6b52-48ee-bf95-da0c2b4ee5b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:21.019613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96240e-6b52-48ee-bf95-da0c2b4ee5b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:21.021196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96240e-6b52-48ee-bf95-da0c2b4ee5b4 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:21.044560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 611/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 00:20:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:21.045758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7fc4d-15cf-4e06-8003-8d651c5ac3b6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:21.154276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7fc4d-15cf-4e06-8003-8d651c5ac3b6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:21.192223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad0b90-e337-4aae-98df-e14348631f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:21.193045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad0b90-e337-4aae-98df-e14348631f42 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:21.193371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad0b90-e337-4aae-98df-e14348631f42 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:21.193696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad0b90-e337-4aae-98df-e14348631f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:21.206606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad0b90-e337-4aae-98df-e14348631f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:21.208062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad0b90-e337-4aae-98df-e14348631f42 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-22ad0b90-e337-4aae-98df-e14348631f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:21.224381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:21.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:21.226601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:21.226601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:21.226601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:21.226601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 617/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:21.231974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96240e-6b52-48ee-bf95-da0c2b4ee5b4 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:21.345902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96240e-6b52-48ee-bf95-da0c2b4ee5b4 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:21.348615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:21.381172 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cabcf-ea89-4a4a-a09c-1c2403fce80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:21.381778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cabcf-ea89-4a4a-a09c-1c2403fce80e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:21.382051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cabcf-ea89-4a4a-a09c-1c2403fce80e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:21.382317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cabcf-ea89-4a4a-a09c-1c2403fce80e None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:21.382952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cabcf-ea89-4a4a-a09c-1c2403fce80e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:21.383249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cabcf-ea89-4a4a-a09c-1c2403fce80e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:21.385173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5cabcf-ea89-4a4a-a09c-1c2403fce80e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:21.396351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 612/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:21.402290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad0b90-e337-4aae-98df-e14348631f42 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:21.413245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:21.521230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad0b90-e337-4aae-98df-e14348631f42 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:21.557393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0131a-eb9e-486e-9a31-c17d2af82b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:21.558014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0131a-eb9e-486e-9a31-c17d2af82b9e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:21.558289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0131a-eb9e-486e-9a31-c17d2af82b9e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:21.558556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0131a-eb9e-486e-9a31-c17d2af82b9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:21.565376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0131a-eb9e-486e-9a31-c17d2af82b9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:21.567017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d0131a-eb9e-486e-9a31-c17d2af82b9e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-e5d0131a-eb9e-486e-9a31-c17d2af82b9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:20:21.580273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:21.581279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:21.582170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:20:21.582170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:20:21.582170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:20:21.582170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:20:21.582170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:20:21.582170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:20:21.582170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:21.582170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:21.582170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 618/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:20:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:21.588379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cabcf-ea89-4a4a-a09c-1c2403fce80e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:21.699656 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cabcf-ea89-4a4a-a09c-1c2403fce80e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:21.736133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694ad71-e100-4ee0-92cf-708e5c1be10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:21.736869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694ad71-e100-4ee0-92cf-708e5c1be10f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:21.737036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694ad71-e100-4ee0-92cf-708e5c1be10f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:21.737319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694ad71-e100-4ee0-92cf-708e5c1be10f None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:21.738046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694ad71-e100-4ee0-92cf-708e5c1be10f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:21.738350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694ad71-e100-4ee0-92cf-708e5c1be10f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:21.739855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694ad71-e100-4ee0-92cf-708e5c1be10f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:21.753132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 613/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:20:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:21.759885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0131a-eb9e-486e-9a31-c17d2af82b9e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:21.844369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:21.890452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0131a-eb9e-486e-9a31-c17d2af82b9e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:21.926712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f89dc-534b-4c5f-a952-4d0e88f77eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:21.927402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f89dc-534b-4c5f-a952-4d0e88f77eaa None admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:21.927681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f89dc-534b-4c5f-a952-4d0e88f77eaa None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:21.927941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f89dc-534b-4c5f-a952-4d0e88f77eaa None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/e30caaea6d104a53a8654442c2383b7e/roles/87ec4f3044a84c9b8459c5458278b04c` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:21.944008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f89dc-534b-4c5f-a952-4d0e88f77eaa None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=e30caaea6d104a53a8654442c2383b7e, role_id=87ec4f3044a84c9b8459c5458278b04c)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:21.945615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f89dc-534b-4c5f-a952-4d0e88f77eaa None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:21.958748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 619/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:20:21 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/e30caaea6d104a53a8654442c2383b7e/roles/87ec4f3044a84c9b8459c5458278b04c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:20:23.930840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3c21f-c2a9-46e9-855d-a4f7b4cdd2f8 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:23.930840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3c21f-c2a9-46e9-855d-a4f7b4cdd2f8 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:23.931635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3c21f-c2a9-46e9-855d-a4f7b4cdd2f8 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:23.932703 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 614/1233] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:23.938139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c21f5-1e9b-4d3e-9023-997d711f5284 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:23.938569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c21f5-1e9b-4d3e-9023-997d711f5284 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:23.938733 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c21f5-1e9b-4d3e-9023-997d711f5284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:23.957134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bf6c21f5-1e9b-4d3e-9023-997d711f5284 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:23.958319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bf6c21f5-1e9b-4d3e-9023-997d711f5284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:24.028686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 620/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:24.313509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3c21f-c2a9-46e9-855d-a4f7b4cdd2f8 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:24.342440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:24.422566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3c21f-c2a9-46e9-855d-a4f7b4cdd2f8 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:24.459015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fcd96-a715-406a-b71c-7b4374295e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:24.459641 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fcd96-a715-406a-b71c-7b4374295e35 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:24.459913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fcd96-a715-406a-b71c-7b4374295e35 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:24.460214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fcd96-a715-406a-b71c-7b4374295e35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:24.471110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fcd96-a715-406a-b71c-7b4374295e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-b07fcd96-a715-406a-b71c-7b4374295e35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:20:24.477196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:24.478742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:24.479610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 615/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 1 00:20:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:24.485565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c21f5-1e9b-4d3e-9023-997d711f5284 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:24.536715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:24.596590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c21f5-1e9b-4d3e-9023-997d711f5284 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:24.636131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb730d-706c-4f09-81a7-c35fc73871e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:24.636791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb730d-706c-4f09-81a7-c35fc73871e3 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:24.637078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb730d-706c-4f09-81a7-c35fc73871e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:24.637381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb730d-706c-4f09-81a7-c35fc73871e3 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:24.637999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb730d-706c-4f09-81a7-c35fc73871e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:24.638298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb730d-706c-4f09-81a7-c35fc73871e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:24.639818 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb730d-706c-4f09-81a7-c35fc73871e3 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:24.650674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 621/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 1 00:20:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:24.657994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fcd96-a715-406a-b71c-7b4374295e35 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:24.751223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:24.770096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fcd96-a715-406a-b71c-7b4374295e35 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:24.818063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1f7ae-ff20-4f89-ad28-e1b9a0c38987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:24.818779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1f7ae-ff20-4f89-ad28-e1b9a0c38987 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:24.819208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1f7ae-ff20-4f89-ad28-e1b9a0c38987 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:24.819497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1f7ae-ff20-4f89-ad28-e1b9a0c38987 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:24.834363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1f7ae-ff20-4f89-ad28-e1b9a0c38987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:24.836181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1f7ae-ff20-4f89-ad28-e1b9a0c38987 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-63e1f7ae-ff20-4f89-ad28-e1b9a0c38987 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:24.853491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:24.854593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:24.855998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:24.855998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:24.855998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:24.855998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 616/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:24.861276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb730d-706c-4f09-81a7-c35fc73871e3 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:24.969408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb730d-706c-4f09-81a7-c35fc73871e3 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:24.979145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:25.005860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5922-d876-4ca7-a3ae-548ea93a1d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:25.006396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5922-d876-4ca7-a3ae-548ea93a1d76 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:25.006675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5922-d876-4ca7-a3ae-548ea93a1d76 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:25.006948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb5922-d876-4ca7-a3ae-548ea93a1d76 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:25.007649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb5922-d876-4ca7-a3ae-548ea93a1d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:25.007910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb5922-d876-4ca7-a3ae-548ea93a1d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:25.009467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb5922-d876-4ca7-a3ae-548ea93a1d76 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:25.021784 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 622/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:25.028405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1f7ae-ff20-4f89-ad28-e1b9a0c38987 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:25.142330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1f7ae-ff20-4f89-ad28-e1b9a0c38987 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:25.183900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441c5f4-09d0-45e5-a5a9-e7854f33d91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:25.184728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441c5f4-09d0-45e5-a5a9-e7854f33d91c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:25.185059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441c5f4-09d0-45e5-a5a9-e7854f33d91c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:25.185651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441c5f4-09d0-45e5-a5a9-e7854f33d91c None admin] PATH_INFO: `/v3/users/placement` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:25.189325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:25.195893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441c5f4-09d0-45e5-a5a9-e7854f33d91c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:25.198076 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441c5f4-09d0-45e5-a5a9-e7854f33d91c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-1441c5f4-09d0-45e5-a5a9-e7854f33d91c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:20:25.211303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:25.213013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:25.214417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 617/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 00:20:25.219383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5922-d876-4ca7-a3ae-548ea93a1d76 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:25.336726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb5922-d876-4ca7-a3ae-548ea93a1d76 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:25.373070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90719-3770-403b-b70c-7398b1bb94cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:25.373685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90719-3770-403b-b70c-7398b1bb94cc None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:25.373962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90719-3770-403b-b70c-7398b1bb94cc None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:25.374236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90719-3770-403b-b70c-7398b1bb94cc None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:25.375025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d90719-3770-403b-b70c-7398b1bb94cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:25.375450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d90719-3770-403b-b70c-7398b1bb94cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:25.377001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d90719-3770-403b-b70c-7398b1bb94cc None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:25.400773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 623/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 00:20:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:25.404282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441c5f4-09d0-45e5-a5a9-e7854f33d91c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:25.531862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441c5f4-09d0-45e5-a5a9-e7854f33d91c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:25.573798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e22397-990c-4acb-b971-27f6d49792c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:25.574496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e22397-990c-4acb-b971-27f6d49792c9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:25.575563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e22397-990c-4acb-b971-27f6d49792c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:25.575563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e22397-990c-4acb-b971-27f6d49792c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:25.646136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:25.655930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e22397-990c-4acb-b971-27f6d49792c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:25.657542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e22397-990c-4acb-b971-27f6d49792c9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:25.772214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 618/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:20:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 00:20:25.779049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90719-3770-403b-b70c-7398b1bb94cc None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:25.811620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:25.902741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90719-3770-403b-b70c-7398b1bb94cc None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:25.938635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996a6fe-47f9-4e92-a21e-caa695536489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:25.939268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996a6fe-47f9-4e92-a21e-caa695536489 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:25.939541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996a6fe-47f9-4e92-a21e-caa695536489 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:25.939809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996a6fe-47f9-4e92-a21e-caa695536489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:25.950482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996a6fe-47f9-4e92-a21e-caa695536489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:25.951934 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996a6fe-47f9-4e92-a21e-caa695536489 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-6996a6fe-47f9-4e92-a21e-caa695536489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:25.966489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:25.967683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:25.968681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:25.968681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:25.968681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:25.972688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 624/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:25.973632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e22397-990c-4acb-b971-27f6d49792c9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:26.079767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:26.086055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e22397-990c-4acb-b971-27f6d49792c9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:26.121804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651699cc-7601-452f-80b3-a1a8575bb71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:26.122403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651699cc-7601-452f-80b3-a1a8575bb71f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:26.122679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651699cc-7601-452f-80b3-a1a8575bb71f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:26.122946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651699cc-7601-452f-80b3-a1a8575bb71f None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:26.123582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651699cc-7601-452f-80b3-a1a8575bb71f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:26.123880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651699cc-7601-452f-80b3-a1a8575bb71f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:26.125275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651699cc-7601-452f-80b3-a1a8575bb71f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:26.141180 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 619/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:26.147914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996a6fe-47f9-4e92-a21e-caa695536489 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:26.238549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:26.264432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996a6fe-47f9-4e92-a21e-caa695536489 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:26.299081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed890df-6b78-4898-ae6c-c01a4bedcc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:26.299787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed890df-6b78-4898-ae6c-c01a4bedcc59 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:26.300066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed890df-6b78-4898-ae6c-c01a4bedcc59 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:26.300338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed890df-6b78-4898-ae6c-c01a4bedcc59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:26.307542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed890df-6b78-4898-ae6c-c01a4bedcc59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:26.309055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed890df-6b78-4898-ae6c-c01a4bedcc59 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-eed890df-6b78-4898-ae6c-c01a4bedcc59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:20:26.320148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:26.321147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:26.322107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:20:26.322107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:20:26.322107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:20:26.322107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:20:26.322107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:20:26.322107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:20:26.322107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:26.322107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:26.322107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 625/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:20:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:26.328228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651699cc-7601-452f-80b3-a1a8575bb71f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:26.446191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651699cc-7601-452f-80b3-a1a8575bb71f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:26.484110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e71cd3-b212-4aef-9b8f-fd06edab96ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:26.484658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e71cd3-b212-4aef-9b8f-fd06edab96ba None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:26.484949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e71cd3-b212-4aef-9b8f-fd06edab96ba None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:26.485199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e71cd3-b212-4aef-9b8f-fd06edab96ba None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:26.485890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e71cd3-b212-4aef-9b8f-fd06edab96ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:26.486196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e71cd3-b212-4aef-9b8f-fd06edab96ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:26.487815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e71cd3-b212-4aef-9b8f-fd06edab96ba None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:26.500487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 620/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:20:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:26.506969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed890df-6b78-4898-ae6c-c01a4bedcc59 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:26.624113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed890df-6b78-4898-ae6c-c01a4bedcc59 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:26.662814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbeb89-2382-454a-a37b-680ff8f47ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:26.663582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbeb89-2382-454a-a37b-680ff8f47ff5 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:26.664053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbeb89-2382-454a-a37b-680ff8f47ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:26.664335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbeb89-2382-454a-a37b-680ff8f47ff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:26.665272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbeb89-2382-454a-a37b-680ff8f47ff5 None admin] RBAC: Adding query filter params (role.id=87ec4f3044a84c9b8459c5458278b04c, scope.project.id=ed93469862874c75951d95e28929c904, user.id=e30caaea6d104a53a8654442c2383b7e) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:26.666126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbeb89-2382-454a-a37b-680ff8f47ff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:26.668023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbeb89-2382-454a-a37b-680ff8f47ff5 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:26.674888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 626/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:20:26 2022] GET /identity/v3/role_assignments?user.id=e30caaea6d104a53a8654442c2383b7e&scope.project.id=ed93469862874c75951d95e28929c904&role.id=87ec4f3044a84c9b8459c5458278b04c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:28.649755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc52f-cdaa-468c-a070-22929c4b7fb0 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:28.650545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc52f-cdaa-468c-a070-22929c4b7fb0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:28.650545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc52f-cdaa-468c-a070-22929c4b7fb0 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:28.651669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 621/1247] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:28.657616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdeaf4-6733-4631-ac50-717ed8cdb941 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:28.658174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdeaf4-6733-4631-ac50-717ed8cdb941 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:28.658174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdeaf4-6733-4631-ac50-717ed8cdb941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:28.666782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:28.682843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bffdeaf4-6733-4631-ac50-717ed8cdb941 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:28.684061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bffdeaf4-6733-4631-ac50-717ed8cdb941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:28.760325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 627/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:29.036674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc52f-cdaa-468c-a070-22929c4b7fb0 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:29.055799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:29.153880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc52f-cdaa-468c-a070-22929c4b7fb0 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:29.191100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea580650-a073-4cff-8c8e-1c6509507dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:29.191739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea580650-a073-4cff-8c8e-1c6509507dd9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:29.192017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea580650-a073-4cff-8c8e-1c6509507dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:29.192324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea580650-a073-4cff-8c8e-1c6509507dd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:29.204821 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea580650-a073-4cff-8c8e-1c6509507dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-ea580650-a073-4cff-8c8e-1c6509507dd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:20:29.211042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:29.212378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:29.213685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 622/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:20:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:29.218679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdeaf4-6733-4631-ac50-717ed8cdb941 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:29.328826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdeaf4-6733-4631-ac50-717ed8cdb941 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:29.364227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:29.371507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ecb90-6a12-453e-ac19-d5e142e887d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:29.372247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ecb90-6a12-453e-ac19-d5e142e887d6 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:29.372543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ecb90-6a12-453e-ac19-d5e142e887d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:29.372816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ecb90-6a12-453e-ac19-d5e142e887d6 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:29.373480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ecb90-6a12-453e-ac19-d5e142e887d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:29.373779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ecb90-6a12-453e-ac19-d5e142e887d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:29.375164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ecb90-6a12-453e-ac19-d5e142e887d6 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:29.385738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 628/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:20:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:29.391593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea580650-a073-4cff-8c8e-1c6509507dd9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:29.476895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:29.503671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea580650-a073-4cff-8c8e-1c6509507dd9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:29.540619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63ac9c-96a3-45af-9ab3-5136b155df8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:29.541363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63ac9c-96a3-45af-9ab3-5136b155df8f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:29.541621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63ac9c-96a3-45af-9ab3-5136b155df8f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:29.541913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63ac9c-96a3-45af-9ab3-5136b155df8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:29.552241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63ac9c-96a3-45af-9ab3-5136b155df8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:29.553737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63ac9c-96a3-45af-9ab3-5136b155df8f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-4f63ac9c-96a3-45af-9ab3-5136b155df8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:29.569486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:29.570594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:29.572095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:29.572095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:29.572095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:29.572095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 623/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:29.577528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ecb90-6a12-453e-ac19-d5e142e887d6 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:29.691020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ecb90-6a12-453e-ac19-d5e142e887d6 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:29.730423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e6e50-64fd-4464-914a-e6c9f8b5f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:29.731027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e6e50-64fd-4464-914a-e6c9f8b5f66d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:29.731304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e6e50-64fd-4464-914a-e6c9f8b5f66d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:29.731574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e6e50-64fd-4464-914a-e6c9f8b5f66d None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:29.732296 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e6e50-64fd-4464-914a-e6c9f8b5f66d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:29.732632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e6e50-64fd-4464-914a-e6c9f8b5f66d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:29.734163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e6e50-64fd-4464-914a-e6c9f8b5f66d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:29.746800 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 629/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:29.752545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63ac9c-96a3-45af-9ab3-5136b155df8f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:29.871055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63ac9c-96a3-45af-9ab3-5136b155df8f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:29.899061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:29.907640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6191652-f584-4c56-b9b7-44e19e363137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:29.908302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6191652-f584-4c56-b9b7-44e19e363137 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:29.908610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6191652-f584-4c56-b9b7-44e19e363137 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:29.908889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6191652-f584-4c56-b9b7-44e19e363137 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:29.918062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6191652-f584-4c56-b9b7-44e19e363137 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:29.919534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6191652-f584-4c56-b9b7-44e19e363137 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-b6191652-f584-4c56-b9b7-44e19e363137 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:20:29.931585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:29.932707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:29.933633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 624/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:29.939126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e6e50-64fd-4464-914a-e6c9f8b5f66d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:30.010352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:30.060403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e6e50-64fd-4464-914a-e6c9f8b5f66d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:30.097303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740eadc-1d72-46fa-95a5-fbd3e00c12bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:30.097994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740eadc-1d72-46fa-95a5-fbd3e00c12bd None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:30.098284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740eadc-1d72-46fa-95a5-fbd3e00c12bd None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:30.098613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740eadc-1d72-46fa-95a5-fbd3e00c12bd None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:30.099346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740eadc-1d72-46fa-95a5-fbd3e00c12bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:30.099660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740eadc-1d72-46fa-95a5-fbd3e00c12bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:30.101310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740eadc-1d72-46fa-95a5-fbd3e00c12bd None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:30.121034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 630/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 00:20:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:20:30.126994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6191652-f584-4c56-b9b7-44e19e363137 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:30.237272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6191652-f584-4c56-b9b7-44e19e363137 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:30.276466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6e74-5de9-4090-ab48-886ee4ad4f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:30.277007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6e74-5de9-4090-ab48-886ee4ad4f74 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:30.277280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6e74-5de9-4090-ab48-886ee4ad4f74 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:30.277542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de6e74-5de9-4090-ab48-886ee4ad4f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:30.336245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:30.352379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de6e74-5de9-4090-ab48-886ee4ad4f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:30.354474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de6e74-5de9-4090-ab48-886ee4ad4f74 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:30.461477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 625/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:20:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:20:30.467959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740eadc-1d72-46fa-95a5-fbd3e00c12bd None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:30.578589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740eadc-1d72-46fa-95a5-fbd3e00c12bd None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:30.593979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:30.616710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a78b48-9543-4ba6-8968-cc9d96ecd346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:30.617347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78b48-9543-4ba6-8968-cc9d96ecd346 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:30.617627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78b48-9543-4ba6-8968-cc9d96ecd346 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:30.617902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a78b48-9543-4ba6-8968-cc9d96ecd346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:30.628043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a78b48-9543-4ba6-8968-cc9d96ecd346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:30.629485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a78b48-9543-4ba6-8968-cc9d96ecd346 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-e5a78b48-9543-4ba6-8968-cc9d96ecd346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:30.645690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:30.646854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:30.648092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:30.648092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:30.648092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:30.648092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 631/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:30.653337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6e74-5de9-4090-ab48-886ee4ad4f74 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:30.747900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:30.762028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de6e74-5de9-4090-ab48-886ee4ad4f74 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:30.803125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10dc51-1add-4136-81e9-47fe3c4ac411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:30.803774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10dc51-1add-4136-81e9-47fe3c4ac411 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:30.804057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10dc51-1add-4136-81e9-47fe3c4ac411 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:30.804531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10dc51-1add-4136-81e9-47fe3c4ac411 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:30.805265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10dc51-1add-4136-81e9-47fe3c4ac411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:30.805588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10dc51-1add-4136-81e9-47fe3c4ac411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:30.807800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10dc51-1add-4136-81e9-47fe3c4ac411 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:30.820306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 626/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:20:30.827190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a78b48-9543-4ba6-8968-cc9d96ecd346 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:30.936207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a78b48-9543-4ba6-8968-cc9d96ecd346 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:30.973587 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26004e01-c769-447b-a73e-61843203671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:30.974212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26004e01-c769-447b-a73e-61843203671d None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:30.974496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26004e01-c769-447b-a73e-61843203671d None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:30.974783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26004e01-c769-447b-a73e-61843203671d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:30.982299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26004e01-c769-447b-a73e-61843203671d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:30.983823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26004e01-c769-447b-a73e-61843203671d None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-26004e01-c769-447b-a73e-61843203671d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:20:30.996667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:30.998225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:30.999255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:20:30.999255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:20:30.999255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:20:30.999255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:20:30.999255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:20:30.999255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:20:30.999255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:30.999255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:30.999255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 632/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:20:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:31.003827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10dc51-1add-4136-81e9-47fe3c4ac411 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:31.117938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10dc51-1add-4136-81e9-47fe3c4ac411 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:31.156133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40912847-5bb3-48e5-a13b-27addff087b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:31.156783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40912847-5bb3-48e5-a13b-27addff087b2 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:31.157112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40912847-5bb3-48e5-a13b-27addff087b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:31.157339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40912847-5bb3-48e5-a13b-27addff087b2 None admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:31.158209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40912847-5bb3-48e5-a13b-27addff087b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:31.158520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40912847-5bb3-48e5-a13b-27addff087b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:31.160191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40912847-5bb3-48e5-a13b-27addff087b2 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:31.181515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 627/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:20:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:31.182837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26004e01-c769-447b-a73e-61843203671d None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:31.213396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:31.296668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26004e01-c769-447b-a73e-61843203671d None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:31.331941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dee5a9-3573-4a71-9587-88a073d5f461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:31.332589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dee5a9-3573-4a71-9587-88a073d5f461 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:31.332867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dee5a9-3573-4a71-9587-88a073d5f461 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:31.333139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dee5a9-3573-4a71-9587-88a073d5f461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:31.333886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dee5a9-3573-4a71-9587-88a073d5f461 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=ed93469862874c75951d95e28929c904, user.id=e30caaea6d104a53a8654442c2383b7e) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:31.334193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dee5a9-3573-4a71-9587-88a073d5f461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:31.335677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dee5a9-3573-4a71-9587-88a073d5f461 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:31.342532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 633/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:20:31 2022] GET /identity/v3/role_assignments?user.id=e30caaea6d104a53a8654442c2383b7e&scope.project.id=ed93469862874c75951d95e28929c904&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:33.275139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6e6f0-e1f1-4ed8-9295-d6e03e768cb6 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:33.275630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6e6f0-e1f1-4ed8-9295-d6e03e768cb6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:33.275862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6e6f0-e1f1-4ed8-9295-d6e03e768cb6 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:33.277341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 628/1261] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:33.282348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a799163-34db-4404-96da-af9136959ba3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:33.282750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a799163-34db-4404-96da-af9136959ba3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:33.282887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a799163-34db-4404-96da-af9136959ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:33.301237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-7a799163-34db-4404-96da-af9136959ba3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:33.302402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7a799163-34db-4404-96da-af9136959ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:33.366782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 634/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:33.636031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6e6f0-e1f1-4ed8-9295-d6e03e768cb6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:33.646853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:33.735574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6e6f0-e1f1-4ed8-9295-d6e03e768cb6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:33.769033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb644478-ef3f-4816-a7f1-ccb4a996d169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:33.769647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb644478-ef3f-4816-a7f1-ccb4a996d169 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:33.769924 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb644478-ef3f-4816-a7f1-ccb4a996d169 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:33.770191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb644478-ef3f-4816-a7f1-ccb4a996d169 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:33.780699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb644478-ef3f-4816-a7f1-ccb4a996d169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-fb644478-ef3f-4816-a7f1-ccb4a996d169 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:20:33.785484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:33.787041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:20:33.788469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:33.791593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 629/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:33.792728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a799163-34db-4404-96da-af9136959ba3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:33.813763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:33.901663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a799163-34db-4404-96da-af9136959ba3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:33.934218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa97dc0-03db-4171-90d0-1e905bbc2397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:33.934810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa97dc0-03db-4171-90d0-1e905bbc2397 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:33.935085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa97dc0-03db-4171-90d0-1e905bbc2397 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:33.935352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa97dc0-03db-4171-90d0-1e905bbc2397 None admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:33.936006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa97dc0-03db-4171-90d0-1e905bbc2397 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:33.936304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa97dc0-03db-4171-90d0-1e905bbc2397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:33.937710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa97dc0-03db-4171-90d0-1e905bbc2397 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:33.945446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 635/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:33.951726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb644478-ef3f-4816-a7f1-ccb4a996d169 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:34.017163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:34.049352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb644478-ef3f-4816-a7f1-ccb4a996d169 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:34.083085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2058a-5074-4b19-b5db-a0bcbcfb323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:34.083691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2058a-5074-4b19-b5db-a0bcbcfb323f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:34.083966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2058a-5074-4b19-b5db-a0bcbcfb323f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:34.084236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2058a-5074-4b19-b5db-a0bcbcfb323f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:34.094363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2058a-5074-4b19-b5db-a0bcbcfb323f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:34.095824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2058a-5074-4b19-b5db-a0bcbcfb323f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-26e2058a-5074-4b19-b5db-a0bcbcfb323f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:34.110365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:34.115867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:34.116914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:34.116914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:34.116914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:34.116914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 630/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:34.117150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa97dc0-03db-4171-90d0-1e905bbc2397 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:34.197694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:34.216612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa97dc0-03db-4171-90d0-1e905bbc2397 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:34.250613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4508a-8f4f-4fd7-bb40-87c1caa9544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:34.251214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4508a-8f4f-4fd7-bb40-87c1caa9544e None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:34.251491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4508a-8f4f-4fd7-bb40-87c1caa9544e None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:34.251761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4508a-8f4f-4fd7-bb40-87c1caa9544e None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:34.252393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4508a-8f4f-4fd7-bb40-87c1caa9544e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:34.252720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4508a-8f4f-4fd7-bb40-87c1caa9544e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:34.254099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4508a-8f4f-4fd7-bb40-87c1caa9544e None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:34.265829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 636/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:34.271491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2058a-5074-4b19-b5db-a0bcbcfb323f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:34.367957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2058a-5074-4b19-b5db-a0bcbcfb323f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:34.387294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:34.401139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8da28-dd6f-4900-882f-1143cf7ba58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:34.401753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8da28-dd6f-4900-882f-1143cf7ba58c None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:34.402025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8da28-dd6f-4900-882f-1143cf7ba58c None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:34.402293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8da28-dd6f-4900-882f-1143cf7ba58c None admin] PATH_INFO: `/v3/users/placement` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:34.409400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8da28-dd6f-4900-882f-1143cf7ba58c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:34.410824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8da28-dd6f-4900-882f-1143cf7ba58c None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-8dd8da28-dd6f-4900-882f-1143cf7ba58c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:20:34.422462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:34.423364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:34.424614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 631/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:34.429293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4508a-8f4f-4fd7-bb40-87c1caa9544e None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:34.533720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4508a-8f4f-4fd7-bb40-87c1caa9544e None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:34.571068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cdbed-b308-462b-a9d7-ff0ba1136ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:34.571900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cdbed-b308-462b-a9d7-ff0ba1136ca0 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:34.572179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cdbed-b308-462b-a9d7-ff0ba1136ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:34.572448 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cdbed-b308-462b-a9d7-ff0ba1136ca0 None admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:34.573531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cdbed-b308-462b-a9d7-ff0ba1136ca0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:34.574086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cdbed-b308-462b-a9d7-ff0ba1136ca0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:34.576021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cdbed-b308-462b-a9d7-ff0ba1136ca0 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:34.586306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:34.595307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 637/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 00:20:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:34.601231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8da28-dd6f-4900-882f-1143cf7ba58c None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:34.696814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8da28-dd6f-4900-882f-1143cf7ba58c None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:34.729672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b13328-2e16-47a0-9bc1-893f5b8ae05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:34.730271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13328-2e16-47a0-9bc1-893f5b8ae05a None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:34.730543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13328-2e16-47a0-9bc1-893f5b8ae05a None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:34.730808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13328-2e16-47a0-9bc1-893f5b8ae05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:34.740930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b13328-2e16-47a0-9bc1-893f5b8ae05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:34.742364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b13328-2e16-47a0-9bc1-893f5b8ae05a None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-62b13328-2e16-47a0-9bc1-893f5b8ae05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:34.756003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:34.757157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:34.758058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:34.758058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:34.758058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:34.758058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 632/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:34.763016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cdbed-b308-462b-a9d7-ff0ba1136ca0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:34.863332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cdbed-b308-462b-a9d7-ff0ba1136ca0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:34.895256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5abee-af1e-4131-9758-11705b5cf15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:34.895845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5abee-af1e-4131-9758-11705b5cf15c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:34.896118 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5abee-af1e-4131-9758-11705b5cf15c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:34.896384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5abee-af1e-4131-9758-11705b5cf15c None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:34.897036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5abee-af1e-4131-9758-11705b5cf15c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:34.897334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5abee-af1e-4131-9758-11705b5cf15c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:34.898686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5abee-af1e-4131-9758-11705b5cf15c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:34.910124 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 638/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:34.916441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b13328-2e16-47a0-9bc1-893f5b8ae05a None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:34.947263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:35.014312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b13328-2e16-47a0-9bc1-893f5b8ae05a None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:35.046609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3b9d2-a59f-4d6b-a7e1-b22b6de67d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:35.047211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3b9d2-a59f-4d6b-a7e1-b22b6de67d0f None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:35.047485 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3b9d2-a59f-4d6b-a7e1-b22b6de67d0f None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:35.047749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3b9d2-a59f-4d6b-a7e1-b22b6de67d0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:35.054077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3b9d2-a59f-4d6b-a7e1-b22b6de67d0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:35.055794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3b9d2-a59f-4d6b-a7e1-b22b6de67d0f None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-31c3b9d2-a59f-4d6b-a7e1-b22b6de67d0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:20:35.066963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:35.068019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:35.069425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:20:35.069425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:20:35.069425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:20:35.069425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:20:35.069425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:20:35.069425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:20:35.069425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:35.069425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:35.069425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 633/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:20:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:35.074326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5abee-af1e-4131-9758-11705b5cf15c None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:35.130848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:35.172561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5abee-af1e-4131-9758-11705b5cf15c None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:35.204921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a07ee-c49f-46e9-8f9f-343ad560b28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:35.205509 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a07ee-c49f-46e9-8f9f-343ad560b28c None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:35.205785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a07ee-c49f-46e9-8f9f-343ad560b28c None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:35.206054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a07ee-c49f-46e9-8f9f-343ad560b28c None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:35.206743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a07ee-c49f-46e9-8f9f-343ad560b28c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:35.207044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a07ee-c49f-46e9-8f9f-343ad560b28c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:35.208458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a07ee-c49f-46e9-8f9f-343ad560b28c None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:35.220629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 639/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:20:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:35.227925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3b9d2-a59f-4d6b-a7e1-b22b6de67d0f None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:35.320934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:35.330467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3b9d2-a59f-4d6b-a7e1-b22b6de67d0f None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:35.362666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df58404-aeeb-4ac3-b203-fd4100d49f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:35.363300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df58404-aeeb-4ac3-b203-fd4100d49f71 None admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:35.363573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df58404-aeeb-4ac3-b203-fd4100d49f71 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:35.363840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df58404-aeeb-4ac3-b203-fd4100d49f71 None admin] PATH_INFO: `/v3/projects/ed93469862874c75951d95e28929c904/users/e30caaea6d104a53a8654442c2383b7e/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:35.365695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df58404-aeeb-4ac3-b203-fd4100d49f71 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed93469862874c75951d95e28929c904, user_id=e30caaea6d104a53a8654442c2383b7e, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:35.367124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df58404-aeeb-4ac3-b203-fd4100d49f71 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:35.379796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 634/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 1 00:20:35 2022] PUT /identity/v3/projects/ed93469862874c75951d95e28929c904/users/e30caaea6d104a53a8654442c2383b7e/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:20:37.242240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e2d5a-ad99-44f0-bd78-be2807b98335 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:37.242624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e2d5a-ad99-44f0-bd78-be2807b98335 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:37.242787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e2d5a-ad99-44f0-bd78-be2807b98335 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:37.244155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 640/1274] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:37.249234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d9fbf-ab16-4865-b364-1a1abb08d9e6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:37.249612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d9fbf-ab16-4865-b364-1a1abb08d9e6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:37.249901 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d9fbf-ab16-4865-b364-1a1abb08d9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:37.268105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-db2d9fbf-ab16-4865-b364-1a1abb08d9e6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:37.269314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-db2d9fbf-ab16-4865-b364-1a1abb08d9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:37.334482 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 635/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:37.602109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e2d5a-ad99-44f0-bd78-be2807b98335 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:37.702062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e2d5a-ad99-44f0-bd78-be2807b98335 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:37.718593 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:37.735420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6971c2d-61f7-4dc0-a581-6f1c24308821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:37.736035 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971c2d-61f7-4dc0-a581-6f1c24308821 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:37.736312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971c2d-61f7-4dc0-a581-6f1c24308821 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:37.736611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6971c2d-61f7-4dc0-a581-6f1c24308821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:37.746960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6971c2d-61f7-4dc0-a581-6f1c24308821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-b6971c2d-61f7-4dc0-a581-6f1c24308821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 00:20:37.751728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:37.752774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:37.753648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 641/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 1 00:20:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:37.759532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d9fbf-ab16-4865-b364-1a1abb08d9e6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:37.840804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:37.859510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d9fbf-ab16-4865-b364-1a1abb08d9e6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:37.892853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2e732-945f-4983-b329-56da3cada519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:37.893475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2e732-945f-4983-b329-56da3cada519 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:37.893757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2e732-945f-4983-b329-56da3cada519 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:37.894029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2e732-945f-4983-b329-56da3cada519 None admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:37.894692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2e732-945f-4983-b329-56da3cada519 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:37.894995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2e732-945f-4983-b329-56da3cada519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:37.896368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2e732-945f-4983-b329-56da3cada519 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:37.904481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 636/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 1 00:20:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:37.910781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6971c2d-61f7-4dc0-a581-6f1c24308821 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:38.007283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6971c2d-61f7-4dc0-a581-6f1c24308821 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:38.039308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720f96d-5adc-44ae-a4f2-e424464b12ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:38.040638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720f96d-5adc-44ae-a4f2-e424464b12ea None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:38.040914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720f96d-5adc-44ae-a4f2-e424464b12ea None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:38.041183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720f96d-5adc-44ae-a4f2-e424464b12ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:38.050701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720f96d-5adc-44ae-a4f2-e424464b12ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:38.052135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720f96d-5adc-44ae-a4f2-e424464b12ea None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-0720f96d-5adc-44ae-a4f2-e424464b12ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:38.067017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:38.068100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:38.069234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:38.069234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:38.069234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:38.069234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 642/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:38.074356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2e732-945f-4983-b329-56da3cada519 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:38.171702 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2e732-945f-4983-b329-56da3cada519 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:38.203583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad64129-7b50-4230-a471-c16e3bf8a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:38.204167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad64129-7b50-4230-a471-c16e3bf8a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:38.204442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad64129-7b50-4230-a471-c16e3bf8a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:38.204738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad64129-7b50-4230-a471-c16e3bf8a8e1 None admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:38.205361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad64129-7b50-4230-a471-c16e3bf8a8e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:38.205655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad64129-7b50-4230-a471-c16e3bf8a8e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:38.207050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad64129-7b50-4230-a471-c16e3bf8a8e1 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:38.218955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 637/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:38.224914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720f96d-5adc-44ae-a4f2-e424464b12ea None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:38.254673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:38.324505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720f96d-5adc-44ae-a4f2-e424464b12ea None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:38.357652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ab34c-c428-4e64-9f21-3d3297e06b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:38.358265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ab34c-c428-4e64-9f21-3d3297e06b05 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:38.358535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ab34c-c428-4e64-9f21-3d3297e06b05 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:38.358799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ab34c-c428-4e64-9f21-3d3297e06b05 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:38.365844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ab34c-c428-4e64-9f21-3d3297e06b05 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:38.367265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ab34c-c428-4e64-9f21-3d3297e06b05 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-7f2ab34c-c428-4e64-9f21-3d3297e06b05 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 00:20:38.379033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:38.380119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:38.381278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 643/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:38.386492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad64129-7b50-4230-a471-c16e3bf8a8e1 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:38.390332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:38.483880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad64129-7b50-4230-a471-c16e3bf8a8e1 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:38.524109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be698e2-b5d7-4113-8150-e6477d028d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:38.524792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be698e2-b5d7-4113-8150-e6477d028d1e None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:38.525073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be698e2-b5d7-4113-8150-e6477d028d1e None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:38.525346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be698e2-b5d7-4113-8150-e6477d028d1e None admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:38.526070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be698e2-b5d7-4113-8150-e6477d028d1e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:38.526373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be698e2-b5d7-4113-8150-e6477d028d1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:38.527807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be698e2-b5d7-4113-8150-e6477d028d1e None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:38.544377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 638/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 1 00:20:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:38.550089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ab34c-c428-4e64-9f21-3d3297e06b05 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:38.640327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:38.650342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ab34c-c428-4e64-9f21-3d3297e06b05 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:38.683331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6cd9-0f42-4601-a0c6-9cfa6273e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:38.683927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6cd9-0f42-4601-a0c6-9cfa6273e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:38.684206 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6cd9-0f42-4601-a0c6-9cfa6273e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:38.684479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6cd9-0f42-4601-a0c6-9cfa6273e7a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:38.750044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6cd9-0f42-4601-a0c6-9cfa6273e7a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:38.751446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6cd9-0f42-4601-a0c6-9cfa6273e7a8 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:38.842427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:38.854381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 644/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 1 00:20:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 00:20:38.860137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be698e2-b5d7-4113-8150-e6477d028d1e None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:38.920679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:38.958170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be698e2-b5d7-4113-8150-e6477d028d1e None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:38.991156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c57f4-d730-4cb1-9010-6be1036438a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:38.991770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c57f4-d730-4cb1-9010-6be1036438a9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:38.992048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c57f4-d730-4cb1-9010-6be1036438a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:38.992318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c57f4-d730-4cb1-9010-6be1036438a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:39.003010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c57f4-d730-4cb1-9010-6be1036438a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:39.004398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c57f4-d730-4cb1-9010-6be1036438a9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-092c57f4-d730-4cb1-9010-6be1036438a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 00:20:39.018681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:39.019606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 00:20:39.020590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 00:20:39.020590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 00:20:39.020590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:39.020590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 639/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:39.025416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6cd9-0f42-4601-a0c6-9cfa6273e7a8 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:39.121328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6cd9-0f42-4601-a0c6-9cfa6273e7a8 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:39.154029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2c2d0-d7d6-4249-a858-0fc0fb4f9993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:39.154623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2c2d0-d7d6-4249-a858-0fc0fb4f9993 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:39.154900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2c2d0-d7d6-4249-a858-0fc0fb4f9993 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:39.155170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2c2d0-d7d6-4249-a858-0fc0fb4f9993 None admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:39.155797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2c2d0-d7d6-4249-a858-0fc0fb4f9993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:39.156094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2c2d0-d7d6-4249-a858-0fc0fb4f9993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:39.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2c2d0-d7d6-4249-a858-0fc0fb4f9993 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:39.168394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 645/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 1 00:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:39.174004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c57f4-d730-4cb1-9010-6be1036438a9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:39.273850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c57f4-d730-4cb1-9010-6be1036438a9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:39.287098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:39.308636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdcf64-1909-440d-babf-101b928c08f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:39.309254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdcf64-1909-440d-babf-101b928c08f9 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:39.309528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdcf64-1909-440d-babf-101b928c08f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:39.309795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdcf64-1909-440d-babf-101b928c08f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:39.316249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdcf64-1909-440d-babf-101b928c08f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:39.317725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cdcf64-1909-440d-babf-101b928c08f9 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application [None req-52cdcf64-1909-440d-babf-101b928c08f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 00:20:39.329225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:39.330242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:39.331359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 00:20:39.331359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 00:20:39.331359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 00:20:39.331359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 00:20:39.331359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 00:20:39.331359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 00:20:39.331359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 00:20:39.331359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: ERROR keystone.server.flask.application Sep 01 00:20:39.331359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 640/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 1 00:20:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:39.336307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2c2d0-d7d6-4249-a858-0fc0fb4f9993 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:39.386929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:39.435833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2c2d0-d7d6-4249-a858-0fc0fb4f9993 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:39.467813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15444657-78bb-43a2-88c8-3f6fb7d42efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:39.468399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15444657-78bb-43a2-88c8-3f6fb7d42efc None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:39.468698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15444657-78bb-43a2-88c8-3f6fb7d42efc None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:39.468963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15444657-78bb-43a2-88c8-3f6fb7d42efc None admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:39.469631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15444657-78bb-43a2-88c8-3f6fb7d42efc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:39.469928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15444657-78bb-43a2-88c8-3f6fb7d42efc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:39.471641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15444657-78bb-43a2-88c8-3f6fb7d42efc None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:39.483811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 646/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 1 00:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:39.489864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdcf64-1909-440d-babf-101b928c08f9 None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:39.585323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cdcf64-1909-440d-babf-101b928c08f9 None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:39.617970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268351d-7a82-41ef-b179-87b15da90588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:39.618553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268351d-7a82-41ef-b179-87b15da90588 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:39.618825 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268351d-7a82-41ef-b179-87b15da90588 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:39.619097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268351d-7a82-41ef-b179-87b15da90588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:39.619792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268351d-7a82-41ef-b179-87b15da90588 None admin] RBAC: Adding query filter params (role.id=1d817eb4d5c2420ea85c0c6ca25df22a, scope.project.id=ed93469862874c75951d95e28929c904, user.id=e30caaea6d104a53a8654442c2383b7e) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:39.620092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268351d-7a82-41ef-b179-87b15da90588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:39.621545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268351d-7a82-41ef-b179-87b15da90588 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:39.627052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 641/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 1 00:20:39 2022] GET /identity/v3/role_assignments?user.id=e30caaea6d104a53a8654442c2383b7e&scope.project.id=ed93469862874c75951d95e28929c904&role.id=1d817eb4d5c2420ea85c0c6ca25df22a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:41.552556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744cfe4-bba1-47b7-a915-37046ed8dca2 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:41.553115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744cfe4-bba1-47b7-a915-37046ed8dca2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:41.553115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744cfe4-bba1-47b7-a915-37046ed8dca2 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:41.554446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 647/1288] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:41.560466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6dda-a9eb-41b5-ade6-200f57f8fc30 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:41.560952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6dda-a9eb-41b5-ade6-200f57f8fc30 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:41.561066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6dda-a9eb-41b5-ade6-200f57f8fc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:41.580139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-9c3d6dda-a9eb-41b5-ade6-200f57f8fc30 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:41.581546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9c3d6dda-a9eb-41b5-ade6-200f57f8fc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:41.610967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:41.648130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 642/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:41.920197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744cfe4-bba1-47b7-a915-37046ed8dca2 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:42.018517 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744cfe4-bba1-47b7-a915-37046ed8dca2 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:42.028832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:42.052323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9aa36-7eca-48d2-b76b-e4eec7a0a541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:42.052964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9aa36-7eca-48d2-b76b-e4eec7a0a541 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:42.053236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9aa36-7eca-48d2-b76b-e4eec7a0a541 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:42.053500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9aa36-7eca-48d2-b76b-e4eec7a0a541 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:42.054069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9aa36-7eca-48d2-b76b-e4eec7a0a541 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:42.055480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9aa36-7eca-48d2-b76b-e4eec7a0a541 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-e8f9aa36-7eca-48d2-b76b-e4eec7a0a541 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:42.063937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:42.065183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:42.066043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 648/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:20:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:42.071080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6dda-a9eb-41b5-ade6-200f57f8fc30 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:42.167978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d6dda-a9eb-41b5-ade6-200f57f8fc30 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:42.200276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d2b3a-bd77-461b-8d0f-99be76cc88bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:42.200839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d2b3a-bd77-461b-8d0f-99be76cc88bf None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:42.201124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d2b3a-bd77-461b-8d0f-99be76cc88bf None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:42.201396 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d2b3a-bd77-461b-8d0f-99be76cc88bf None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:42.202022 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d2b3a-bd77-461b-8d0f-99be76cc88bf None admin] RBAC: Adding query filter params (name=placement) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:42.202324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d2b3a-bd77-461b-8d0f-99be76cc88bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:42.203709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d2b3a-bd77-461b-8d0f-99be76cc88bf None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:42.208378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 643/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 1 00:20:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:42.214571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9aa36-7eca-48d2-b76b-e4eec7a0a541 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:42.313698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9aa36-7eca-48d2-b76b-e4eec7a0a541 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:42.346989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf59b8-e8fa-46a4-a7c4-d1a367b6eb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:42.347587 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf59b8-e8fa-46a4-a7c4-d1a367b6eb98 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:42.347867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf59b8-e8fa-46a4-a7c4-d1a367b6eb98 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:42.348142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf59b8-e8fa-46a4-a7c4-d1a367b6eb98 None admin] PATH_INFO: `/v3/services` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:42.348854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf59b8-e8fa-46a4-a7c4-d1a367b6eb98 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:42.349158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf59b8-e8fa-46a4-a7c4-d1a367b6eb98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:42.350536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf59b8-e8fa-46a4-a7c4-d1a367b6eb98 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:42.355795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 649/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 1 00:20:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:44.228688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de784b29-6379-4790-b734-88d02cfb3ca1 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:44.229218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de784b29-6379-4790-b734-88d02cfb3ca1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:44.229218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de784b29-6379-4790-b734-88d02cfb3ca1 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:44.230770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 644/1293] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:44.235794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c18620-8979-48d1-8b79-7bc671fe7b04 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:44.236194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c18620-8979-48d1-8b79-7bc671fe7b04 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:44.236334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c18620-8979-48d1-8b79-7bc671fe7b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:44.254569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-c1c18620-8979-48d1-8b79-7bc671fe7b04 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:44.255762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c1c18620-8979-48d1-8b79-7bc671fe7b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:44.285845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:44.323792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 650/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:44.589774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de784b29-6379-4790-b734-88d02cfb3ca1 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:44.620787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:44.688330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de784b29-6379-4790-b734-88d02cfb3ca1 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:44.720592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126deefe-7e36-469b-9c08-a261ae6a41b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:44.721431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126deefe-7e36-469b-9c08-a261ae6a41b8 None admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:44.721705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126deefe-7e36-469b-9c08-a261ae6a41b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:44.721971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126deefe-7e36-469b-9c08-a261ae6a41b8 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:44.722611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126deefe-7e36-469b-9c08-a261ae6a41b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:44.723968 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126deefe-7e36-469b-9c08-a261ae6a41b8 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:44.738009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 645/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 1 00:20:44 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:20:46.645856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7356d3-9cae-4d63-b5be-c81047858db2 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:46.646427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7356d3-9cae-4d63-b5be-c81047858db2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:46.646427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7356d3-9cae-4d63-b5be-c81047858db2 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:46.647730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 651/1296] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:46.653329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ecfdd-9bd3-4b7b-8f4d-f408b1c59475 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:46.653749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ecfdd-9bd3-4b7b-8f4d-f408b1c59475 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:46.653874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ecfdd-9bd3-4b7b-8f4d-f408b1c59475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:46.672889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-cb6ecfdd-9bd3-4b7b-8f4d-f408b1c59475 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:46.674088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-cb6ecfdd-9bd3-4b7b-8f4d-f408b1c59475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:46.745596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 646/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:47.013673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7356d3-9cae-4d63-b5be-c81047858db2 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:47.111576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7356d3-9cae-4d63-b5be-c81047858db2 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:47.143702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c7d3-a7ff-41f2-a231-be32bf4308a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:47.144303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c7d3-a7ff-41f2-a231-be32bf4308a0 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:47.144602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c7d3-a7ff-41f2-a231-be32bf4308a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:47.144871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c7d3-a7ff-41f2-a231-be32bf4308a0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:47.145439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47c7d3-a7ff-41f2-a231-be32bf4308a0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:47.146811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47c7d3-a7ff-41f2-a231-be32bf4308a0 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-6f47c7d3-a7ff-41f2-a231-be32bf4308a0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:47.155307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:47.156567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:47.157478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 652/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:20:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:47.162266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ecfdd-9bd3-4b7b-8f4d-f408b1c59475 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:47.181068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:47.261355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ecfdd-9bd3-4b7b-8f4d-f408b1c59475 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:47.294345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5c739-aff0-4281-9e43-92bb9057f34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:47.294947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5c739-aff0-4281-9e43-92bb9057f34b None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:47.295223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5c739-aff0-4281-9e43-92bb9057f34b None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:47.295492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5c739-aff0-4281-9e43-92bb9057f34b None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:47.296114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5c739-aff0-4281-9e43-92bb9057f34b None admin] RBAC: Adding query filter params (name=placement) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:47.296418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5c739-aff0-4281-9e43-92bb9057f34b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:47.297811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5c739-aff0-4281-9e43-92bb9057f34b None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:47.302781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 647/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 1 00:20:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:47.308843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c7d3-a7ff-41f2-a231-be32bf4308a0 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:47.340159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:47.406679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c7d3-a7ff-41f2-a231-be32bf4308a0 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:47.438966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25f38d-e0bf-4d4f-8b46-066bd084920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:47.439554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25f38d-e0bf-4d4f-8b46-066bd084920a None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:47.439831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25f38d-e0bf-4d4f-8b46-066bd084920a None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:47.440102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25f38d-e0bf-4d4f-8b46-066bd084920a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:47.440820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25f38d-e0bf-4d4f-8b46-066bd084920a None admin] RBAC: Adding query filter params (interface=public, service_id=9219e2e45a7149919ff8fcdee15425d8, region_id=RegionOne) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:47.441126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25f38d-e0bf-4d4f-8b46-066bd084920a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:47.442495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25f38d-e0bf-4d4f-8b46-066bd084920a None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:47.447429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 653/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 1 00:20:47 2022] GET /identity/v3/endpoints?service_id=9219e2e45a7149919ff8fcdee15425d8&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:47.453063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5c739-aff0-4281-9e43-92bb9057f34b None admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:47.533086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:47.551367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5c739-aff0-4281-9e43-92bb9057f34b None admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:47.583168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a091a5-3950-4f25-b335-036716b02770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:47.583760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a091a5-3950-4f25-b335-036716b02770 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:47.584038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a091a5-3950-4f25-b335-036716b02770 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:47.584310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a091a5-3950-4f25-b335-036716b02770 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:47.584943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a091a5-3950-4f25-b335-036716b02770 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:47.586830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a091a5-3950-4f25-b335-036716b02770 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:47.593406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 648/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 1 00:20:47 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:20:49.465770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb785b-1e4a-4c62-9be7-6be22b4153ce None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:49.466312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb785b-1e4a-4c62-9be7-6be22b4153ce None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:49.466312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb785b-1e4a-4c62-9be7-6be22b4153ce None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:49.467632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 654/1302] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:20:49.472872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b98434-e960-4085-a20a-c4836d334b6e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:49.473252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b98434-e960-4085-a20a-c4836d334b6e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:49.473423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b98434-e960-4085-a20a-c4836d334b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:49.491898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-75b98434-e960-4085-a20a-c4836d334b6e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:20:49.493973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-75b98434-e960-4085-a20a-c4836d334b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:20:49.558260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 649/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:20:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 00:20:49.826224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb785b-1e4a-4c62-9be7-6be22b4153ce None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:49.918549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:49.925387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb785b-1e4a-4c62-9be7-6be22b4153ce None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:49.969523 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771b5d2-2e91-4c39-87bd-dc90e7933656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:49.970138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771b5d2-2e91-4c39-87bd-dc90e7933656 None admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:49.970410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771b5d2-2e91-4c39-87bd-dc90e7933656 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:49.970675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771b5d2-2e91-4c39-87bd-dc90e7933656 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:49.971509 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771b5d2-2e91-4c39-87bd-dc90e7933656 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:49.972912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771b5d2-2e91-4c39-87bd-dc90e7933656 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application [None req-c771b5d2-2e91-4c39-87bd-dc90e7933656 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 00:20:49.982106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 00:20:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application with Lock( Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self._enter() Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application return self.creator() Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application created_value = creator( Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: ERROR keystone.server.flask.application Sep 01 00:20:49.984354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 655/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 1 00:20:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 00:20:49.989182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b98434-e960-4085-a20a-c4836d334b6e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:50.063279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:20:50.085858 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b98434-e960-4085-a20a-c4836d334b6e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:50.119564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035ee8d-cce5-465f-a2f7-07762ff505a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:50.120161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035ee8d-cce5-465f-a2f7-07762ff505a1 None admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:50.120438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035ee8d-cce5-465f-a2f7-07762ff505a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:50.120735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035ee8d-cce5-465f-a2f7-07762ff505a1 None admin] PATH_INFO: `/v3/services` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:50.121356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035ee8d-cce5-465f-a2f7-07762ff505a1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:20:50.121654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035ee8d-cce5-465f-a2f7-07762ff505a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:50.123018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035ee8d-cce5-465f-a2f7-07762ff505a1 None admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:50.128073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 650/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 1 00:20:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:20:50.133944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771b5d2-2e91-4c39-87bd-dc90e7933656 None admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:20:50.233084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771b5d2-2e91-4c39-87bd-dc90e7933656 None admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:20:50.267343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4bc26-a11e-4251-8246-381edf83d186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:20:50.267950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4bc26-a11e-4251-8246-381edf83d186 None admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:20:50.268227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4bc26-a11e-4251-8246-381edf83d186 None admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:20:50.268516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4bc26-a11e-4251-8246-381edf83d186 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:20:50.269192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4bc26-a11e-4251-8246-381edf83d186 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:20:50.270557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4bc26-a11e-4251-8246-381edf83d186 None admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:20:50.289826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 656/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:20:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:21:00.125366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=30 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:21:00.285408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:21:53.295604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2aa336-040e-47b2-b4f1-65e94cbc6f65 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:53.296171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2aa336-040e-47b2-b4f1-65e94cbc6f65 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:53.296171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2aa336-040e-47b2-b4f1-65e94cbc6f65 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:53.301494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 651/1307] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:21:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:21:53.302993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb7566-2a4b-4378-9841-00a68b22e27d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:53.303438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb7566-2a4b-4378-9841-00a68b22e27d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:53.303543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb7566-2a4b-4378-9841-00a68b22e27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:53.383822 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-33cb7566-2a4b-4378-9841-00a68b22e27d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:21:53.385069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-33cb7566-2a4b-4378-9841-00a68b22e27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:21:53.495344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 657/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:21:53.799702 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7af20-7703-4743-9401-675ca7578a27 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:53.800186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7af20-7703-4743-9401-675ca7578a27 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:53.800640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7af20-7703-4743-9401-675ca7578a27 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:53.803320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 652/1309] 10.222.0.25 () {54 vars in 965 bytes} [Thu Sep 1 00:21:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:21:53.810981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0982c-f86f-47a7-b801-bae2875882e3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:53.811590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0982c-f86f-47a7-b801-bae2875882e3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:53.812216 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0982c-f86f-47a7-b801-bae2875882e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:53.935342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-96f0982c-f86f-47a7-b801-bae2875882e3 None None] MFA Rules not processed for user `cdf97e9e0340441c87316ce03d46cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:21:53.937350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-96f0982c-f86f-47a7-b801-bae2875882e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:21:53.961076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:21:54.022143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 658/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Sep 1 00:21:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:21:54.031106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7af20-7703-4743-9401-675ca7578a27 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:21:54.032850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-85e7af20-7703-4743-9401-675ca7578a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:21:54.157291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7af20-7703-4743-9401-675ca7578a27 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:21:54.252811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a85b2-8005-42f6-8483-d0e87462944f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:21:54.253448 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a85b2-8005-42f6-8483-d0e87462944f service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:54.253731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a85b2-8005-42f6-8483-d0e87462944f service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:54.254005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a85b2-8005-42f6-8483-d0e87462944f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:54.256312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-790a85b2-8005-42f6-8483-d0e87462944f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:21:54.321855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:21:54.388654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a85b2-8005-42f6-8483-d0e87462944f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:21:54.390282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a85b2-8005-42f6-8483-d0e87462944f service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:21:54.411939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 653/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:21:56.757647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dacefc-b1fb-46ce-bc35-ebafc2bd81f7 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:56.758201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dacefc-b1fb-46ce-bc35-ebafc2bd81f7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:56.758201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dacefc-b1fb-46ce-bc35-ebafc2bd81f7 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:56.760649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 659/1312] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:21:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:21:56.766153 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3935b-249a-41de-aef9-d6b414b9af03 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:56.766676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3935b-249a-41de-aef9-d6b414b9af03 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:56.767404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3935b-249a-41de-aef9-d6b414b9af03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:56.869768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-bbb3935b-249a-41de-aef9-d6b414b9af03 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:21:56.871039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-bbb3935b-249a-41de-aef9-d6b414b9af03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:21:56.988718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 654/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:21:57.268885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dacefc-b1fb-46ce-bc35-ebafc2bd81f7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:21:57.270570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a8dacefc-b1fb-46ce-bc35-ebafc2bd81f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:21:57.374525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dacefc-b1fb-46ce-bc35-ebafc2bd81f7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:21:57.447162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:21:57.449970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03b0b5-f3c5-4597-81f8-5f06ebc22a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:21:57.450775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03b0b5-f3c5-4597-81f8-5f06ebc22a00 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:57.451098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03b0b5-f3c5-4597-81f8-5f06ebc22a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:57.451568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03b0b5-f3c5-4597-81f8-5f06ebc22a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:57.453506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-cf03b0b5-f3c5-4597-81f8-5f06ebc22a00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:21:57.565207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03b0b5-f3c5-4597-81f8-5f06ebc22a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:21:57.567203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03b0b5-f3c5-4597-81f8-5f06ebc22a00 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:21:57.586218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 660/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:21:59.826958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a2c26-cae5-4d90-8ec1-20bea17d8e51 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:59.827518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a2c26-cae5-4d90-8ec1-20bea17d8e51 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:59.827518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a2c26-cae5-4d90-8ec1-20bea17d8e51 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:59.828880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 655/1315] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:21:59.835260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8367e-b1aa-4f1d-9763-55187730b1d4 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:21:59.835668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8367e-b1aa-4f1d-9763-55187730b1d4 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:21:59.835808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8367e-b1aa-4f1d-9763-55187730b1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:21:59.900881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-11e8367e-b1aa-4f1d-9763-55187730b1d4 None None] MFA Rules not processed for user `1a50181ffd4545db88d27e0331cd3f67`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:21:59.902094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-11e8367e-b1aa-4f1d-9763-55187730b1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:21:59.953962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:21:59.998695 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 661/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:21:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:00.283142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a2c26-cae5-4d90-8ec1-20bea17d8e51 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:00.284743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b16a2c26-cae5-4d90-8ec1-20bea17d8e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:00.295223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:00.389874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a2c26-cae5-4d90-8ec1-20bea17d8e51 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:00.463969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3f693-14e6-423b-8e20-9ecb4af784ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:00.464598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f693-14e6-423b-8e20-9ecb4af784ce service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:00.464878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f693-14e6-423b-8e20-9ecb4af784ce service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:00.465193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f693-14e6-423b-8e20-9ecb4af784ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:00.466802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a5a3f693-14e6-423b-8e20-9ecb4af784ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:00.528706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:00.575619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3f693-14e6-423b-8e20-9ecb4af784ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:00.577103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3f693-14e6-423b-8e20-9ecb4af784ce service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:00.596102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 656/1317] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:03.052452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4c32a-56af-4da7-b00b-bf7705961a58 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:03.052989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4c32a-56af-4da7-b00b-bf7705961a58 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:03.053083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4c32a-56af-4da7-b00b-bf7705961a58 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:03.054362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 662/1318] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:03.059514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d4f21-9630-4aab-9231-5ccf1d898d82 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:03.059893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d4f21-9630-4aab-9231-5ccf1d898d82 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:03.060058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d4f21-9630-4aab-9231-5ccf1d898d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:03.126453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-4f5d4f21-9630-4aab-9231-5ccf1d898d82 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:03.127652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4f5d4f21-9630-4aab-9231-5ccf1d898d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:03.222653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 657/1319] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:03.506514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4c32a-56af-4da7-b00b-bf7705961a58 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:03.508066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f6b4c32a-56af-4da7-b00b-bf7705961a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:03.613513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4c32a-56af-4da7-b00b-bf7705961a58 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:03.684250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aeae1d-bb1b-4cd6-b82f-6531f9d648e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:03.684872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aeae1d-bb1b-4cd6-b82f-6531f9d648e3 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:03.685146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aeae1d-bb1b-4cd6-b82f-6531f9d648e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:03.685413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aeae1d-bb1b-4cd6-b82f-6531f9d648e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:03.686996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-56aeae1d-bb1b-4cd6-b82f-6531f9d648e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:03.694766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:03.798143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aeae1d-bb1b-4cd6-b82f-6531f9d648e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:03.799610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aeae1d-bb1b-4cd6-b82f-6531f9d648e3 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:03.823826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 663/1320] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:06.367170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fe026-0fed-4d5d-9c5d-96f3d7df361f None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:06.367729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fe026-0fed-4d5d-9c5d-96f3d7df361f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:06.367729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fe026-0fed-4d5d-9c5d-96f3d7df361f None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:06.369118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 658/1321] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:06.374371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182da2e2-238b-475d-9c8a-f277b2faa38f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:06.374782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182da2e2-238b-475d-9c8a-f277b2faa38f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:06.374919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182da2e2-238b-475d-9c8a-f277b2faa38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:06.440702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-182da2e2-238b-475d-9c8a-f277b2faa38f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:06.442045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-182da2e2-238b-475d-9c8a-f277b2faa38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:06.500221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:06.542669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 664/1322] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:06.841358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538791b-cd0d-4967-b458-96f8faed2ad8 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:06.841643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538791b-cd0d-4967-b458-96f8faed2ad8 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:06.841913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538791b-cd0d-4967-b458-96f8faed2ad8 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:06.843825 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 659/1323] 10.222.0.25 () {54 vars in 965 bytes} [Thu Sep 1 00:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:06.850099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed488acd-27a5-464d-ae88-1dfb5b2a0e7f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:06.850409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed488acd-27a5-464d-ae88-1dfb5b2a0e7f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:06.850682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed488acd-27a5-464d-ae88-1dfb5b2a0e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:06.928317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ed488acd-27a5-464d-ae88-1dfb5b2a0e7f None None] MFA Rules not processed for user `cdf97e9e0340441c87316ce03d46cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:06.929578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ed488acd-27a5-464d-ae88-1dfb5b2a0e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:07.013286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 665/1324] 10.222.0.25 () {58 vars in 1062 bytes} [Thu Sep 1 00:22:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:07.022505 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538791b-cd0d-4967-b458-96f8faed2ad8 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:07.024036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b538791b-cd0d-4967-b458-96f8faed2ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:07.043508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:07.133257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538791b-cd0d-4967-b458-96f8faed2ad8 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:07.212807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9d076-e371-4f5f-8566-6166a1646439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:07.213424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d076-e371-4f5f-8566-6166a1646439 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:07.213698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d076-e371-4f5f-8566-6166a1646439 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:07.213963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d076-e371-4f5f-8566-6166a1646439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:07.215618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-84e9d076-e371-4f5f-8566-6166a1646439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:07.291551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:07.330659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9d076-e371-4f5f-8566-6166a1646439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:07.332726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9d076-e371-4f5f-8566-6166a1646439 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:07.351707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 660/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:09.545771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ea894-c33c-464f-a239-903f39b13473 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:09.546293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ea894-c33c-464f-a239-903f39b13473 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:09.546388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ea894-c33c-464f-a239-903f39b13473 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:09.547685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 666/1326] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:09.554237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b0fd9-d95b-4c9a-8dde-d7fe9222ea42 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:09.554704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b0fd9-d95b-4c9a-8dde-d7fe9222ea42 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:09.554914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b0fd9-d95b-4c9a-8dde-d7fe9222ea42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:09.622256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-611b0fd9-d95b-4c9a-8dde-d7fe9222ea42 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:09.623468 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-611b0fd9-d95b-4c9a-8dde-d7fe9222ea42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:09.724844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 661/1327] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:10.009990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ea894-c33c-464f-a239-903f39b13473 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:10.011614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-970ea894-c33c-464f-a239-903f39b13473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:10.073573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:10.118406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ea894-c33c-464f-a239-903f39b13473 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:10.193457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7289d-ffb1-4440-ae6a-d7797229eeef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:10.194068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7289d-ffb1-4440-ae6a-d7797229eeef service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:10.194348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7289d-ffb1-4440-ae6a-d7797229eeef service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:10.194620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7289d-ffb1-4440-ae6a-d7797229eeef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:10.196234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d1c7289d-ffb1-4440-ae6a-d7797229eeef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:10.306160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:10.308530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7289d-ffb1-4440-ae6a-d7797229eeef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:10.310722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7289d-ffb1-4440-ae6a-d7797229eeef service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:10.329015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 667/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:12.292939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0988a-caaa-4174-8e5d-698a3a9cad09 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:12.293408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0988a-caaa-4174-8e5d-698a3a9cad09 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:12.293501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0988a-caaa-4174-8e5d-698a3a9cad09 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:12.298894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 662/1329] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:22:12.307191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff0086-a359-4c14-a5df-733758e1696b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:12.308003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff0086-a359-4c14-a5df-733758e1696b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:12.308475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff0086-a359-4c14-a5df-733758e1696b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:12.386327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b6ff0086-a359-4c14-a5df-733758e1696b None None] MFA Rules not processed for user `1a50181ffd4545db88d27e0331cd3f67`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:12.387549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b6ff0086-a359-4c14-a5df-733758e1696b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:12.484257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 668/1330] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:12.772250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0988a-caaa-4174-8e5d-698a3a9cad09 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:12.773840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fac0988a-caaa-4174-8e5d-698a3a9cad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:12.806004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:12.886256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0988a-caaa-4174-8e5d-698a3a9cad09 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:12.969662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de40b3-c450-4ad7-b1c9-ac7acc82e54b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:12.970276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de40b3-c450-4ad7-b1c9-ac7acc82e54b service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:12.970552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de40b3-c450-4ad7-b1c9-ac7acc82e54b service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:12.970824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de40b3-c450-4ad7-b1c9-ac7acc82e54b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:12.973063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-54de40b3-c450-4ad7-b1c9-ac7acc82e54b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:13.062691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:13.102190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de40b3-c450-4ad7-b1c9-ac7acc82e54b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:13.103681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de40b3-c450-4ad7-b1c9-ac7acc82e54b service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:13.123460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 663/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:15.429883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9dad4-6826-426b-abe6-310570d3aecd None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:15.430438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9dad4-6826-426b-abe6-310570d3aecd None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:15.430438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9dad4-6826-426b-abe6-310570d3aecd None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:15.431783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 669/1332] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:15.436916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18d9b5-36d5-45ad-929e-1d48a7263f21 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:15.437333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18d9b5-36d5-45ad-929e-1d48a7263f21 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:15.437446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18d9b5-36d5-45ad-929e-1d48a7263f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:15.502590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-5f18d9b5-36d5-45ad-929e-1d48a7263f21 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:15.504395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5f18d9b5-36d5-45ad-929e-1d48a7263f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:15.600505 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 664/1333] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:15.882836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9dad4-6826-426b-abe6-310570d3aecd None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:15.884360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9ac9dad4-6826-426b-abe6-310570d3aecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:15.952934 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:15.986929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9dad4-6826-426b-abe6-310570d3aecd None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:16.059965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de248da0-96d4-4940-aeb8-bf33326deba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:16.060584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de248da0-96d4-4940-aeb8-bf33326deba9 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:16.060857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de248da0-96d4-4940-aeb8-bf33326deba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:16.061120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de248da0-96d4-4940-aeb8-bf33326deba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:16.062702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-de248da0-96d4-4940-aeb8-bf33326deba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:16.170507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de248da0-96d4-4940-aeb8-bf33326deba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:16.171880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de248da0-96d4-4940-aeb8-bf33326deba9 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:16.182475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:16.192282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 670/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:18.158653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca080d01-b63c-4d6d-8d01-f525679e4db6 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:18.159264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca080d01-b63c-4d6d-8d01-f525679e4db6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:18.159264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca080d01-b63c-4d6d-8d01-f525679e4db6 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:18.161246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 665/1335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:18.166211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526cba0-67ac-4a02-a1d3-9aa29f697070 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:18.166610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526cba0-67ac-4a02-a1d3-9aa29f697070 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:18.166753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526cba0-67ac-4a02-a1d3-9aa29f697070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:18.233793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6526cba0-67ac-4a02-a1d3-9aa29f697070 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:18.235005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6526cba0-67ac-4a02-a1d3-9aa29f697070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:18.332446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 671/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:18.619428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca080d01-b63c-4d6d-8d01-f525679e4db6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:18.621026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ca080d01-b63c-4d6d-8d01-f525679e4db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:18.660319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:18.723213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca080d01-b63c-4d6d-8d01-f525679e4db6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:18.795508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ce30e-4f25-4b1e-bca6-b08056af0fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:18.796108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ce30e-4f25-4b1e-bca6-b08056af0fac service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:18.796381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ce30e-4f25-4b1e-bca6-b08056af0fac service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:18.796681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ce30e-4f25-4b1e-bca6-b08056af0fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:18.798256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-452ce30e-4f25-4b1e-bca6-b08056af0fac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:18.879671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:18.905241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ce30e-4f25-4b1e-bca6-b08056af0fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:18.906677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ce30e-4f25-4b1e-bca6-b08056af0fac service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:18.924701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 666/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:21.281594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf4720-f774-4b51-a0b9-4b3727fa5a37 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:21.282092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf4720-f774-4b51-a0b9-4b3727fa5a37 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:21.282092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf4720-f774-4b51-a0b9-4b3727fa5a37 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:21.283422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 672/1338] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:21.288642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874861c2-9152-4ad0-b3db-ed6362802c2a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:21.289061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874861c2-9152-4ad0-b3db-ed6362802c2a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:21.289222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874861c2-9152-4ad0-b3db-ed6362802c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:21.354442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-874861c2-9152-4ad0-b3db-ed6362802c2a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:21.355635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-874861c2-9152-4ad0-b3db-ed6362802c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:21.451438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 667/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:21.732924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf4720-f774-4b51-a0b9-4b3727fa5a37 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:21.734466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-33cf4720-f774-4b51-a0b9-4b3727fa5a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:21.812942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:21.840721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf4720-f774-4b51-a0b9-4b3727fa5a37 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:21.916193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3097a-e4e7-4ae6-b2ce-c3aecb7e4e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:21.916840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3097a-e4e7-4ae6-b2ce-c3aecb7e4e37 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:21.917120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3097a-e4e7-4ae6-b2ce-c3aecb7e4e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:21.917391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3097a-e4e7-4ae6-b2ce-c3aecb7e4e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:21.919019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2da3097a-e4e7-4ae6-b2ce-c3aecb7e4e37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:22.026028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3097a-e4e7-4ae6-b2ce-c3aecb7e4e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:22.027399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3097a-e4e7-4ae6-b2ce-c3aecb7e4e37 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:22.046054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 673/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:25.973744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50bf31-6a5c-4048-a205-de21a49a9b98 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:25.974286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50bf31-6a5c-4048-a205-de21a49a9b98 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:25.974286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50bf31-6a5c-4048-a205-de21a49a9b98 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:25.975813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 668/1341] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:25.980774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1c85-2018-4d81-9ff7-0c4775f80ac6 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:25.981208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1c85-2018-4d81-9ff7-0c4775f80ac6 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:25.981356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1c85-2018-4d81-9ff7-0c4775f80ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:25.986906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:26.055620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-7e7f1c85-2018-4d81-9ff7-0c4775f80ac6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:26.057010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7e7f1c85-2018-4d81-9ff7-0c4775f80ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:26.165900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 674/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:26.446831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50bf31-6a5c-4048-a205-de21a49a9b98 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:26.448415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0e50bf31-6a5c-4048-a205-de21a49a9b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:26.496455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:26.551873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50bf31-6a5c-4048-a205-de21a49a9b98 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:26.624738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9b0ff-36e5-437b-a9f9-eb8113ba0a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:26.625339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b0ff-36e5-437b-a9f9-eb8113ba0a02 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:26.625615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b0ff-36e5-437b-a9f9-eb8113ba0a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:26.625878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9b0ff-36e5-437b-a9f9-eb8113ba0a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:26.627447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-36c9b0ff-36e5-437b-a9f9-eb8113ba0a02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:26.720783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:26.738891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9b0ff-36e5-437b-a9f9-eb8113ba0a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:26.740345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9b0ff-36e5-437b-a9f9-eb8113ba0a02 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:26.759004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 669/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:29.278785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dd7de-9c0c-4e7d-bfbd-02d14bf5a322 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:29.279325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dd7de-9c0c-4e7d-bfbd-02d14bf5a322 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:29.279325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89dd7de-9c0c-4e7d-bfbd-02d14bf5a322 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:29.280685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 675/1344] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:29.286815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47705302-2d09-4d4f-9888-24c83accecad None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:29.287358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47705302-2d09-4d4f-9888-24c83accecad None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:29.287358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47705302-2d09-4d4f-9888-24c83accecad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:29.357856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-47705302-2d09-4d4f-9888-24c83accecad None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:29.359095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-47705302-2d09-4d4f-9888-24c83accecad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:29.463144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 670/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:29.755101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89dd7de-9c0c-4e7d-bfbd-02d14bf5a322 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:29.756687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a89dd7de-9c0c-4e7d-bfbd-02d14bf5a322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:29.849218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:29.867308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89dd7de-9c0c-4e7d-bfbd-02d14bf5a322 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:29.942002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b7d39-a610-4162-8eb2-034fb089b563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:29.942604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b7d39-a610-4162-8eb2-034fb089b563 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:29.942874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b7d39-a610-4162-8eb2-034fb089b563 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:29.943141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b7d39-a610-4162-8eb2-034fb089b563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:29.944751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d21b7d39-a610-4162-8eb2-034fb089b563 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:30.061349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b7d39-a610-4162-8eb2-034fb089b563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:30.063438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b7d39-a610-4162-8eb2-034fb089b563 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:30.084283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 676/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:33.967560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579515b-a04b-40bd-b925-5797b2c82228 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:33.968013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579515b-a04b-40bd-b925-5797b2c82228 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:33.968180 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579515b-a04b-40bd-b925-5797b2c82228 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:33.973297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 671/1347] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:33.975613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee31da-a9ec-481a-bfad-2d60317cfcf6 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:33.976057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee31da-a9ec-481a-bfad-2d60317cfcf6 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:33.976150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee31da-a9ec-481a-bfad-2d60317cfcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:33.996539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:34.058436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-37ee31da-a9ec-481a-bfad-2d60317cfcf6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:34.059679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-37ee31da-a9ec-481a-bfad-2d60317cfcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:34.165946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 677/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:34.454211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579515b-a04b-40bd-b925-5797b2c82228 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:34.456375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6579515b-a04b-40bd-b925-5797b2c82228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:34.519541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:34.566287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579515b-a04b-40bd-b925-5797b2c82228 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:34.641290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363bf20-c323-462d-ac16-7505acab88d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:34.641883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363bf20-c323-462d-ac16-7505acab88d1 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:34.642159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363bf20-c323-462d-ac16-7505acab88d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:34.642437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363bf20-c323-462d-ac16-7505acab88d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:34.644726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8363bf20-c323-462d-ac16-7505acab88d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:34.750940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:34.755320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363bf20-c323-462d-ac16-7505acab88d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:34.756974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8363bf20-c323-462d-ac16-7505acab88d1 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:34.775762 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 672/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:39.746344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba7853-d3ff-49f8-a2a0-d0cb54e540d8 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:39.746813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba7853-d3ff-49f8-a2a0-d0cb54e540d8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:39.746892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba7853-d3ff-49f8-a2a0-d0cb54e540d8 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:39.748565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 678/1350] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:39.754194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a8cd8-d121-4932-9c71-52ea44563193 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:39.754623 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a8cd8-d121-4932-9c71-52ea44563193 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:39.754803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a8cd8-d121-4932-9c71-52ea44563193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:39.852289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-050a8cd8-d121-4932-9c71-52ea44563193 None None] MFA Rules not processed for user `e30caaea6d104a53a8654442c2383b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:39.854233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-050a8cd8-d121-4932-9c71-52ea44563193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:40.006942 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 673/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:22:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:22:44.164026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=59 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:44.186579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056f0e9-9108-4933-b494-dc97c815524b None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:44.187095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056f0e9-9108-4933-b494-dc97c815524b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:44.187539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056f0e9-9108-4933-b494-dc97c815524b None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:44.189166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 679/1352] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:22:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:44.195304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985bcca-438a-4d10-b796-eeda3f69fcd8 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:44.195802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985bcca-438a-4d10-b796-eeda3f69fcd8 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:44.195953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985bcca-438a-4d10-b796-eeda3f69fcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:44.262710 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:44.291682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-4985bcca-438a-4d10-b796-eeda3f69fcd8 None None] MFA Rules not processed for user `e30caaea6d104a53a8654442c2383b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:44.293023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4985bcca-438a-4d10-b796-eeda3f69fcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:44.409040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 674/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:22:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:45.825632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5972b1-23de-4948-983c-87fc7003f293 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:45.826187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5972b1-23de-4948-983c-87fc7003f293 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:45.826187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5972b1-23de-4948-983c-87fc7003f293 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:45.827509 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 680/1354] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:45.833565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f6899-399c-4605-8c66-e69a5692bc8f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:45.834020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f6899-399c-4605-8c66-e69a5692bc8f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:45.834259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f6899-399c-4605-8c66-e69a5692bc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:45.911542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-7d2f6899-399c-4605-8c66-e69a5692bc8f None None] MFA Rules not processed for user `e30caaea6d104a53a8654442c2383b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:45.912771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7d2f6899-399c-4605-8c66-e69a5692bc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:45.954877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:46.022514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 675/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:22:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:49.333225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bdce4-4b5b-473b-ac2f-3ced40579b92 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:49.333808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bdce4-4b5b-473b-ac2f-3ced40579b92 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:49.333808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bdce4-4b5b-473b-ac2f-3ced40579b92 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:49.335086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 681/1356] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:49.340273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689f5c9-10ce-4492-aacd-82f5c4b3e80c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:49.340726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689f5c9-10ce-4492-aacd-82f5c4b3e80c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:49.340863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689f5c9-10ce-4492-aacd-82f5c4b3e80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:49.427289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-d689f5c9-10ce-4492-aacd-82f5c4b3e80c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:49.428509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d689f5c9-10ce-4492-aacd-82f5c4b3e80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:49.565487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 676/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:49.867795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b5cff-8501-43fa-ac03-b72aae892688 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:49.868311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b5cff-8501-43fa-ac03-b72aae892688 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:49.868752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b5cff-8501-43fa-ac03-b72aae892688 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:49.870264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 682/1358] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 1 00:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:49.881773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c5e19-e2ca-40c1-8dd6-4aeeff0a2553 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:49.883025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c5e19-e2ca-40c1-8dd6-4aeeff0a2553 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:49.883412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c5e19-e2ca-40c1-8dd6-4aeeff0a2553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:49.903750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:49.983260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-6f3c5e19-e2ca-40c1-8dd6-4aeeff0a2553 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:49.984899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6f3c5e19-e2ca-40c1-8dd6-4aeeff0a2553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:50.124390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 677/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 1 00:22:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:50.132518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b5cff-8501-43fa-ac03-b72aae892688 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:50.134234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ac6b5cff-8501-43fa-ac03-b72aae892688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:50.305602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b5cff-8501-43fa-ac03-b72aae892688 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:50.400290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:50.444896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a3b18-8de4-4c7a-93ee-c3f6d6868593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:50.445591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a3b18-8de4-4c7a-93ee-c3f6d6868593 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:50.445938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a3b18-8de4-4c7a-93ee-c3f6d6868593 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:50.446276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a3b18-8de4-4c7a-93ee-c3f6d6868593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:50.448469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6c8a3b18-8de4-4c7a-93ee-c3f6d6868593 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:50.547726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a3b18-8de4-4c7a-93ee-c3f6d6868593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:50.549887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a3b18-8de4-4c7a-93ee-c3f6d6868593 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:50.568968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 683/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:22:52.738778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97593cd0-1af0-4630-8682-5f919e7783d7 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:52.739357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97593cd0-1af0-4630-8682-5f919e7783d7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:52.739357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97593cd0-1af0-4630-8682-5f919e7783d7 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:52.740876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 678/1361] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:52.745865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c7f72-6d95-4b79-972b-d5eb202f8142 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:52.746258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c7f72-6d95-4b79-972b-d5eb202f8142 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:52.746419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c7f72-6d95-4b79-972b-d5eb202f8142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:52.818396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ac4c7f72-6d95-4b79-972b-d5eb202f8142 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:52.819850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ac4c7f72-6d95-4b79-972b-d5eb202f8142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:52.869493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:52.948642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 684/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:53.238579 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97593cd0-1af0-4630-8682-5f919e7783d7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:53.240201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-97593cd0-1af0-4630-8682-5f919e7783d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:53.316254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:53.408970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97593cd0-1af0-4630-8682-5f919e7783d7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:53.535753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ca42d-0fda-4ad8-9e83-94183f9361ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:53.536363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ca42d-0fda-4ad8-9e83-94183f9361ee service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:53.536677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ca42d-0fda-4ad8-9e83-94183f9361ee service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:53.536946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ca42d-0fda-4ad8-9e83-94183f9361ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:53.538554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b90ca42d-0fda-4ad8-9e83-94183f9361ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:53.562808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:53.641229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ca42d-0fda-4ad8-9e83-94183f9361ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:53.642894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ca42d-0fda-4ad8-9e83-94183f9361ee service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:53.662336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 679/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:55.934147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11676033-35d4-4db3-b57d-bfd3be97201b None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:55.934714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11676033-35d4-4db3-b57d-bfd3be97201b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:55.934714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11676033-35d4-4db3-b57d-bfd3be97201b None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:55.936043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 685/1364] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:55.941889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f8250-ee12-480f-b8cc-e4700937a1bc None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:55.942365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f8250-ee12-480f-b8cc-e4700937a1bc None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:55.942462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f8250-ee12-480f-b8cc-e4700937a1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:56.013919 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-009f8250-ee12-480f-b8cc-e4700937a1bc None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:56.015515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-009f8250-ee12-480f-b8cc-e4700937a1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:56.121600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:56.131957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 680/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:22:56.419123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710dcad-9c40-4418-b716-9e1cb8276377 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:56.419530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710dcad-9c40-4418-b716-9e1cb8276377 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:56.419928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710dcad-9c40-4418-b716-9e1cb8276377 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:56.421575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 686/1366] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 1 00:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:56.427594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3be6ab-e012-4f63-b296-3a3ea4ff5a1a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:56.427893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3be6ab-e012-4f63-b296-3a3ea4ff5a1a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:56.428169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3be6ab-e012-4f63-b296-3a3ea4ff5a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:56.520865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0b3be6ab-e012-4f63-b296-3a3ea4ff5a1a None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:56.522155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0b3be6ab-e012-4f63-b296-3a3ea4ff5a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:56.656001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 681/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 1 00:22:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:56.664416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710dcad-9c40-4418-b716-9e1cb8276377 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:56.666135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6710dcad-9c40-4418-b716-9e1cb8276377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:56.850224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710dcad-9c40-4418-b716-9e1cb8276377 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:56.872451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:56.976507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1fff2-c313-41d2-8e43-6f58c17f805f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:56.977354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1fff2-c313-41d2-8e43-6f58c17f805f service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:56.977735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1fff2-c313-41d2-8e43-6f58c17f805f service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:56.978110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1fff2-c313-41d2-8e43-6f58c17f805f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:56.979961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0db1fff2-c313-41d2-8e43-6f58c17f805f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:57.084981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1fff2-c313-41d2-8e43-6f58c17f805f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:57.086609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db1fff2-c313-41d2-8e43-6f58c17f805f service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:57.107899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 687/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:22:57.592259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f67c0-bebb-4135-8615-2e8628213480 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:57.593222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f67c0-bebb-4135-8615-2e8628213480 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:57.593222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f67c0-bebb-4135-8615-2e8628213480 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:57.595576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 682/1369] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:57.600439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de4201-394b-40ba-93ac-a9e8d9ab073f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:57.601261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de4201-394b-40ba-93ac-a9e8d9ab073f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:57.601261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de4201-394b-40ba-93ac-a9e8d9ab073f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:57.638640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:57.684455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a1de4201-394b-40ba-93ac-a9e8d9ab073f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:57.686064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a1de4201-394b-40ba-93ac-a9e8d9ab073f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:57.803319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 688/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:58.434286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9af8-5091-4a4f-b27b-c52e962d3af7 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:58.434757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9af8-5091-4a4f-b27b-c52e962d3af7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:58.435182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9af8-5091-4a4f-b27b-c52e962d3af7 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:58.436707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 683/1371] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 1 00:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:58.442190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62393bf-c01e-469d-af11-7b66bd77b457 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:58.442966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62393bf-c01e-469d-af11-7b66bd77b457 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:58.443411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62393bf-c01e-469d-af11-7b66bd77b457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:58.527047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f62393bf-c01e-469d-af11-7b66bd77b457 None None] MFA Rules not processed for user `bb5590d8961d47aa846117a236730d93`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:58.528773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f62393bf-c01e-469d-af11-7b66bd77b457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:58.580532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:58.614458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 689/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 1 00:22:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:58.622225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a9af8-5091-4a4f-b27b-c52e962d3af7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:58.623894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-850a9af8-5091-4a4f-b27b-c52e962d3af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:58.677705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:58.743749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a9af8-5091-4a4f-b27b-c52e962d3af7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:58.868050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6df857-2650-40e3-8ce1-a0b4a8434b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:58.868830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6df857-2650-40e3-8ce1-a0b4a8434b9f service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:58.869238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6df857-2650-40e3-8ce1-a0b4a8434b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:58.869667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6df857-2650-40e3-8ce1-a0b4a8434b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:58.871619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ac6df857-2650-40e3-8ce1-a0b4a8434b9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:59.000560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:59.017427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6df857-2650-40e3-8ce1-a0b4a8434b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:22:59.019043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6df857-2650-40e3-8ce1-a0b4a8434b9f service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:22:59.047770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 684/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:22:59.178212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187e1e0-fba5-418d-be65-3ca4082863d9 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:59.178501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187e1e0-fba5-418d-be65-3ca4082863d9 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:59.178778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187e1e0-fba5-418d-be65-3ca4082863d9 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:59.180171 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 690/1374] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:22:59.185653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef19f2d-070f-42ac-97da-86e3e656ceaa None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:59.185944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef19f2d-070f-42ac-97da-86e3e656ceaa None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:59.186224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef19f2d-070f-42ac-97da-86e3e656ceaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:59.262068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fef19f2d-070f-42ac-97da-86e3e656ceaa None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:22:59.263500 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fef19f2d-070f-42ac-97da-86e3e656ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:59.367839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 685/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:22:59.656780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187e1e0-fba5-418d-be65-3ca4082863d9 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:22:59.658897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8187e1e0-fba5-418d-be65-3ca4082863d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:22:59.813620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187e1e0-fba5-418d-be65-3ca4082863d9 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:22:59.870447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:22:59.934461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ef26c-1062-4213-9c74-73eaf636c822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:22:59.934984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ef26c-1062-4213-9c74-73eaf636c822 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:22:59.935257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ef26c-1062-4213-9c74-73eaf636c822 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:22:59.935520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ef26c-1062-4213-9c74-73eaf636c822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:22:59.937136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-239ef26c-1062-4213-9c74-73eaf636c822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:00.029575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ef26c-1062-4213-9c74-73eaf636c822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:00.031188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ef26c-1062-4213-9c74-73eaf636c822 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:00.050208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 691/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:02.037380 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448dcba-d74e-4915-aeb7-ea07f580a623 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:02.037922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448dcba-d74e-4915-aeb7-ea07f580a623 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:02.037922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448dcba-d74e-4915-aeb7-ea07f580a623 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:02.039300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 686/1377] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:02.044361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4fbc0-dd30-435e-8f58-6b08a3f5f27f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:02.044847 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4fbc0-dd30-435e-8f58-6b08a3f5f27f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:02.045290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4fbc0-dd30-435e-8f58-6b08a3f5f27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:02.134036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3df4fbc0-dd30-435e-8f58-6b08a3f5f27f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:02.135169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3df4fbc0-dd30-435e-8f58-6b08a3f5f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:02.142409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:02.238187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 692/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:02.534899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448dcba-d74e-4915-aeb7-ea07f580a623 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:02.537432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d448dcba-d74e-4915-aeb7-ea07f580a623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:02.744120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:02.839128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448dcba-d74e-4915-aeb7-ea07f580a623 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:02.960259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa4107-04a4-4cab-b200-05e2f2341d93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:02.961048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa4107-04a4-4cab-b200-05e2f2341d93 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:02.961437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa4107-04a4-4cab-b200-05e2f2341d93 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:02.961804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa4107-04a4-4cab-b200-05e2f2341d93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:02.963482 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ccfa4107-04a4-4cab-b200-05e2f2341d93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:02.984231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:03.059926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa4107-04a4-4cab-b200-05e2f2341d93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:03.061650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa4107-04a4-4cab-b200-05e2f2341d93 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:03.080822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 687/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:05.062503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12300b9-dcfe-4799-a533-bb4645e93667 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:05.063544 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12300b9-dcfe-4799-a533-bb4645e93667 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:05.063544 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12300b9-dcfe-4799-a533-bb4645e93667 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:05.065381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 693/1380] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:05.070531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec6b2e-4e49-4583-b5da-3062e7329939 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:05.070965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec6b2e-4e49-4583-b5da-3062e7329939 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:05.071058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec6b2e-4e49-4583-b5da-3062e7329939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:05.141093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-00ec6b2e-4e49-4583-b5da-3062e7329939 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:05.142789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-00ec6b2e-4e49-4583-b5da-3062e7329939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:05.246900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:05.256556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 688/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:05.548321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12300b9-dcfe-4799-a533-bb4645e93667 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:05.550058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d12300b9-dcfe-4799-a533-bb4645e93667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:05.698225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:05.710759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12300b9-dcfe-4799-a533-bb4645e93667 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:05.840286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78194031-5647-4dd2-a56c-ed12aa129262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:05.840918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78194031-5647-4dd2-a56c-ed12aa129262 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:05.841196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78194031-5647-4dd2-a56c-ed12aa129262 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:05.841464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78194031-5647-4dd2-a56c-ed12aa129262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:05.843380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-78194031-5647-4dd2-a56c-ed12aa129262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:05.945923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78194031-5647-4dd2-a56c-ed12aa129262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:05.947548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78194031-5647-4dd2-a56c-ed12aa129262 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:05.959703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:05.969337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 694/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:08.214177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62110c9-e924-4b6b-b7b2-9e8dba7c8038 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:08.214718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62110c9-e924-4b6b-b7b2-9e8dba7c8038 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:08.214843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62110c9-e924-4b6b-b7b2-9e8dba7c8038 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:08.216160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 689/1383] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:23:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:08.222090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281aa54-68a1-4aa9-8aaf-6ae48d7d9edb None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:08.222539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281aa54-68a1-4aa9-8aaf-6ae48d7d9edb None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:08.222661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281aa54-68a1-4aa9-8aaf-6ae48d7d9edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:08.293383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-2281aa54-68a1-4aa9-8aaf-6ae48d7d9edb None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:08.294740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2281aa54-68a1-4aa9-8aaf-6ae48d7d9edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:08.402422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 695/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:08.689840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62110c9-e924-4b6b-b7b2-9e8dba7c8038 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:08.691519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f62110c9-e924-4b6b-b7b2-9e8dba7c8038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:08.855213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62110c9-e924-4b6b-b7b2-9e8dba7c8038 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:08.941328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:08.994851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f1b39-2a6c-4968-bdab-2ffe4da1069f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:08.995462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f1b39-2a6c-4968-bdab-2ffe4da1069f service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:08.995749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f1b39-2a6c-4968-bdab-2ffe4da1069f service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:08.996020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f1b39-2a6c-4968-bdab-2ffe4da1069f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:08.997705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9d5f1b39-2a6c-4968-bdab-2ffe4da1069f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:09.093395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f1b39-2a6c-4968-bdab-2ffe4da1069f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:09.096303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f1b39-2a6c-4968-bdab-2ffe4da1069f service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:09.119607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 690/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:11.111503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0598e32-372a-4578-b5c2-a744d84e2470 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:11.112008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0598e32-372a-4578-b5c2-a744d84e2470 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:11.112008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0598e32-372a-4578-b5c2-a744d84e2470 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:11.113415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 696/1386] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:11.119040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e210096-753a-42ee-9b1f-f3fddffb8dd1 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:11.119469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e210096-753a-42ee-9b1f-f3fddffb8dd1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:11.119671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e210096-753a-42ee-9b1f-f3fddffb8dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:11.191321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1e210096-753a-42ee-9b1f-f3fddffb8dd1 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:11.192625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1e210096-753a-42ee-9b1f-f3fddffb8dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:11.228289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:11.300116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 691/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:11.605207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0598e32-372a-4578-b5c2-a744d84e2470 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:11.606849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f0598e32-372a-4578-b5c2-a744d84e2470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:11.688795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:11.765317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0598e32-372a-4578-b5c2-a744d84e2470 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:11.921238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52429d1d-2e13-4bee-8bfe-14ab7e013280 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:11.921897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52429d1d-2e13-4bee-8bfe-14ab7e013280 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:11.922844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52429d1d-2e13-4bee-8bfe-14ab7e013280 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:11.923376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52429d1d-2e13-4bee-8bfe-14ab7e013280 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:11.925010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-52429d1d-2e13-4bee-8bfe-14ab7e013280 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:11.973592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:12.029600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52429d1d-2e13-4bee-8bfe-14ab7e013280 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:12.031261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52429d1d-2e13-4bee-8bfe-14ab7e013280 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:12.051671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 697/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:14.047742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9046f1b-2a05-450a-a64e-e5f34d9134a6 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:14.048681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9046f1b-2a05-450a-a64e-e5f34d9134a6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:14.048681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9046f1b-2a05-450a-a64e-e5f34d9134a6 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:14.049753 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 692/1389] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:23:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:14.055183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b7b6a-31f0-445b-820a-7d3ab49640d1 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:14.055624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b7b6a-31f0-445b-820a-7d3ab49640d1 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:14.055721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b7b6a-31f0-445b-820a-7d3ab49640d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:14.135657 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a70b7b6a-31f0-445b-820a-7d3ab49640d1 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:14.137223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a70b7b6a-31f0-445b-820a-7d3ab49640d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:14.248151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 698/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:14.559041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9046f1b-2a05-450a-a64e-e5f34d9134a6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:14.560645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a9046f1b-2a05-450a-a64e-e5f34d9134a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:14.722268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9046f1b-2a05-450a-a64e-e5f34d9134a6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:14.739675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:14.844952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f864c-e985-41b5-a4cc-90b684676f6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:14.845679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f864c-e985-41b5-a4cc-90b684676f6f service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:14.846023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f864c-e985-41b5-a4cc-90b684676f6f service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:14.846372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f864c-e985-41b5-a4cc-90b684676f6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:14.848181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-cb7f864c-e985-41b5-a4cc-90b684676f6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:14.954982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f864c-e985-41b5-a4cc-90b684676f6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:14.958102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f864c-e985-41b5-a4cc-90b684676f6f service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:14.983876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 693/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:23:16.964725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a25f33-4394-415b-b18f-8e6907eff02c None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:16.965788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a25f33-4394-415b-b18f-8e6907eff02c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:16.966067 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a25f33-4394-415b-b18f-8e6907eff02c None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:16.967719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 699/1392] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:23:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:16.974234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbb32c-ca08-4bcf-a894-cd7222f21a3b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:16.974666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbb32c-ca08-4bcf-a894-cd7222f21a3b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:16.974800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbb32c-ca08-4bcf-a894-cd7222f21a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:17.000796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:17.042056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-f4fbb32c-ca08-4bcf-a894-cd7222f21a3b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:17.043251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f4fbb32c-ca08-4bcf-a894-cd7222f21a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:17.139250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 694/1393] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:17.427904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a25f33-4394-415b-b18f-8e6907eff02c None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:17.429502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d1a25f33-4394-415b-b18f-8e6907eff02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:17.443793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:17.582239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a25f33-4394-415b-b18f-8e6907eff02c None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:17.681716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:17.720335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776aebdc-bf32-4645-8731-fe37419dbbbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:17.721212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776aebdc-bf32-4645-8731-fe37419dbbbc service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:17.721647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776aebdc-bf32-4645-8731-fe37419dbbbc service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:17.721928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776aebdc-bf32-4645-8731-fe37419dbbbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:17.724153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-776aebdc-bf32-4645-8731-fe37419dbbbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:17.772613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc338b-dae3-4269-8e75-20c8b0928b66 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:17.773162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc338b-dae3-4269-8e75-20c8b0928b66 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:17.773422 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc338b-dae3-4269-8e75-20c8b0928b66 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:17.775227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 695/1394] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:23:17.780158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93ff1-e76c-403a-b751-161343cffc8e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:17.780526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93ff1-e76c-403a-b751-161343cffc8e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:17.780796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93ff1-e76c-403a-b751-161343cffc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:17.818747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776aebdc-bf32-4645-8731-fe37419dbbbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:17.820353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776aebdc-bf32-4645-8731-fe37419dbbbc service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:17.838188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 700/1395] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:17.850711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0ad93ff1-e76c-403a-b751-161343cffc8e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:17.851904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0ad93ff1-e76c-403a-b751-161343cffc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:17.905438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:17.962077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 696/1396] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:23:18.284246 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28097bf-085b-470b-8e09-1875ad614e3f None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:18.284771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28097bf-085b-470b-8e09-1875ad614e3f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:18.284882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28097bf-085b-470b-8e09-1875ad614e3f None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:18.286144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 701/1397] 10.222.0.25 () {54 vars in 967 bytes} [Thu Sep 1 00:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:18.291064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd3c68-8447-45c9-ba3b-c0a78e3d93f2 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:18.291469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd3c68-8447-45c9-ba3b-c0a78e3d93f2 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:18.291603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd3c68-8447-45c9-ba3b-c0a78e3d93f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:18.370644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-55bd3c68-8447-45c9-ba3b-c0a78e3d93f2 None None] MFA Rules not processed for user `52470b620abd4cd29ab7e4752f285bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:18.371863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-55bd3c68-8447-45c9-ba3b-c0a78e3d93f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:18.447975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 697/1398] 10.222.0.25 () {58 vars in 1064 bytes} [Thu Sep 1 00:23:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:18.455574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28097bf-085b-470b-8e09-1875ad614e3f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:18.457141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f28097bf-085b-470b-8e09-1875ad614e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:18.559929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:18.562030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28097bf-085b-470b-8e09-1875ad614e3f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:18.636452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a339b-e583-4d80-aa87-383167da87a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52470b620abd4cd29ab7e4752f285bb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:18.637079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a339b-e583-4d80-aa87-383167da87a5 service glance] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:18.637359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a339b-e583-4d80-aa87-383167da87a5 service glance] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:18.637628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a339b-e583-4d80-aa87-383167da87a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:18.639240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8e9a339b-e583-4d80-aa87-383167da87a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:18.749135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a339b-e583-4d80-aa87-383167da87a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:18.751261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a339b-e583-4d80-aa87-383167da87a5 service glance] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:18.770918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 702/1399] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:19.524484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd493f7d-7d85-4569-8b47-93891eebcd03 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:19.525027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd493f7d-7d85-4569-8b47-93891eebcd03 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:19.525126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd493f7d-7d85-4569-8b47-93891eebcd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:19.585946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:19.595611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-bd493f7d-7d85-4569-8b47-93891eebcd03 None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:19.596841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-bd493f7d-7d85-4569-8b47-93891eebcd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:19.683231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 698/1400] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:23:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:19.728907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728ff8a-9d9b-4159-a704-a1459be70ba5 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:19.729402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728ff8a-9d9b-4159-a704-a1459be70ba5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:19.729493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728ff8a-9d9b-4159-a704-a1459be70ba5 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:19.730794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 703/1401] 10.222.0.25 () {54 vars in 943 bytes} [Thu Sep 1 00:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:19.737878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b58ab-a644-4e26-9088-a9a9009f74ea None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:19.738170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b58ab-a644-4e26-9088-a9a9009f74ea None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:19.738479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b58ab-a644-4e26-9088-a9a9009f74ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:19.807171 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0ce35-55a0-464a-b9f6-9b6d66a277fd None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:19.807449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0ce35-55a0-464a-b9f6-9b6d66a277fd None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:19.807721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0ce35-55a0-464a-b9f6-9b6d66a277fd None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:19.809121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 704/1402] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:23:19.814724 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ba1d-9c19-479a-ad3b-9d73c9f42477 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:19.815006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ba1d-9c19-479a-ad3b-9d73c9f42477 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:19.815278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ba1d-9c19-479a-ad3b-9d73c9f42477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:19.819242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-be4b58ab-a644-4e26-9088-a9a9009f74ea None None] MFA Rules not processed for user `43d6c2a533e849cab240257cc1450c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:19.820461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-be4b58ab-a644-4e26-9088-a9a9009f74ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:19.851373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:19.882962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e415ba1d-9c19-479a-ad3b-9d73c9f42477 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:19.884198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e415ba1d-9c19-479a-ad3b-9d73c9f42477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:19.894861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:19.933999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 699/1403] 10.222.0.25 () {58 vars in 1040 bytes} [Thu Sep 1 00:23:19 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:23:19.944249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b58ab-a644-4e26-9088-a9a9009f74ea None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:19.953720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b58ab-a644-4e26-9088-a9a9009f74ea None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:19.992532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 705/1404] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:20.061956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135abb1d-dfca-439d-a3bf-e753d0bb4903 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:20.062551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135abb1d-dfca-439d-a3bf-e753d0bb4903 service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:20.062832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135abb1d-dfca-439d-a3bf-e753d0bb4903 service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:20.063115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135abb1d-dfca-439d-a3bf-e753d0bb4903 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:20.064779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-135abb1d-dfca-439d-a3bf-e753d0bb4903 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:20.145188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:20.147409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135abb1d-dfca-439d-a3bf-e753d0bb4903 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:20.149342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135abb1d-dfca-439d-a3bf-e753d0bb4903 service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:20.158290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 700/1405] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:23:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 00:23:20.287548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ba1d-9c19-479a-ad3b-9d73c9f42477 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:20.288906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e415ba1d-9c19-479a-ad3b-9d73c9f42477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:20.415868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:20.420024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ba1d-9c19-479a-ad3b-9d73c9f42477 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:20.552664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdca72a-38d8-4f5e-8c12-aac8fd3170ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:20.553275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdca72a-38d8-4f5e-8c12-aac8fd3170ce service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:20.553560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdca72a-38d8-4f5e-8c12-aac8fd3170ce service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:20.553809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdca72a-38d8-4f5e-8c12-aac8fd3170ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:20.555395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8fdca72a-38d8-4f5e-8c12-aac8fd3170ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:20.657359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdca72a-38d8-4f5e-8c12-aac8fd3170ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:20.659398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdca72a-38d8-4f5e-8c12-aac8fd3170ce service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:20.680797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 706/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:22.662021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61ff38-9516-417f-9216-541aabe426b0 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:22.662980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61ff38-9516-417f-9216-541aabe426b0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:22.663292 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61ff38-9516-417f-9216-541aabe426b0 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:22.664885 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 701/1407] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:22.670226 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b2a78-3035-41f7-af32-703235b35659 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:22.670643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b2a78-3035-41f7-af32-703235b35659 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:22.670843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b2a78-3035-41f7-af32-703235b35659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:22.675861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:22.750653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-8d9b2a78-3035-41f7-af32-703235b35659 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:22.751884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8d9b2a78-3035-41f7-af32-703235b35659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:22.864174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 707/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:23.166468 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61ff38-9516-417f-9216-541aabe426b0 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:23.168040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-cd61ff38-9516-417f-9216-541aabe426b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:23.368950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61ff38-9516-417f-9216-541aabe426b0 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:23.444887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:23.500189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb123a13-8820-46b4-96f4-482724a4614a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:23.500903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb123a13-8820-46b4-96f4-482724a4614a service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:23.501176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb123a13-8820-46b4-96f4-482724a4614a service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:23.501444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb123a13-8820-46b4-96f4-482724a4614a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:23.503224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fb123a13-8820-46b4-96f4-482724a4614a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:23.604507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb123a13-8820-46b4-96f4-482724a4614a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:23.606229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb123a13-8820-46b4-96f4-482724a4614a service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:23.627213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 702/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:23:32.862013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=64 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:33.623719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=64 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:42.700262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4b1f1-1802-445f-a130-469e719b91f5 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:42.700859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4b1f1-1802-445f-a130-469e719b91f5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:42.700859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4b1f1-1802-445f-a130-469e719b91f5 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:42.703067 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 708/1410] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 1 00:23:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:42.708252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae435cf-6b30-4040-9dcb-096b022c2b15 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:42.708662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae435cf-6b30-4040-9dcb-096b022c2b15 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:42.708819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae435cf-6b30-4040-9dcb-096b022c2b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:42.787979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-4ae435cf-6b30-4040-9dcb-096b022c2b15 None None] MFA Rules not processed for user `e30caaea6d104a53a8654442c2383b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:42.789207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4ae435cf-6b30-4040-9dcb-096b022c2b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:42.895546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 703/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 1 00:23:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:42.910116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae17ec5-e390-4fc9-abd1-47ed7853c131 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:42.910432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae17ec5-e390-4fc9-abd1-47ed7853c131 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:42.910706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae17ec5-e390-4fc9-abd1-47ed7853c131 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:42.912031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 709/1412] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 1 00:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:42.916461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e658f-33cb-478b-a8e7-663c354c6085 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:42.916771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e658f-33cb-478b-a8e7-663c354c6085 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:42.917044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e658f-33cb-478b-a8e7-663c354c6085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:42.994327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:42.996650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-422e658f-33cb-478b-a8e7-663c354c6085 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:42.998412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-422e658f-33cb-478b-a8e7-663c354c6085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:43.114884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 704/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 1 00:23:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:23:45.462916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2eada-0917-4854-a072-4c485f2efc35 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:45.463441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2eada-0917-4854-a072-4c485f2efc35 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:45.463532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2eada-0917-4854-a072-4c485f2efc35 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:45.464841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:45.470792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76e2d6-1a1e-40fb-8072-2275202fbe88 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:45.471223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76e2d6-1a1e-40fb-8072-2275202fbe88 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:45.471326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76e2d6-1a1e-40fb-8072-2275202fbe88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:45.540363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ba76e2d6-1a1e-40fb-8072-2275202fbe88 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:45.541614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ba76e2d6-1a1e-40fb-8072-2275202fbe88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:45.599263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:45.641082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:45.928398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2eada-0917-4854-a072-4c485f2efc35 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:45.930042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-62a2eada-0917-4854-a072-4c485f2efc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:46.077628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2eada-0917-4854-a072-4c485f2efc35 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:46.150244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:46.190963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c9dd0-1e5e-4676-80bb-b877279e7906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:46.191498 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c9dd0-1e5e-4676-80bb-b877279e7906 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:46.191781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c9dd0-1e5e-4676-80bb-b877279e7906 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:46.192053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c9dd0-1e5e-4676-80bb-b877279e7906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:46.193689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-cd6c9dd0-1e5e-4676-80bb-b877279e7906 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:46.286250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c9dd0-1e5e-4676-80bb-b877279e7906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:46.287899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c9dd0-1e5e-4676-80bb-b877279e7906 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:46.312620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 711/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:47.203913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a450a9-bf38-47a7-bbfb-48e630e8f3e3 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:47.204460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a450a9-bf38-47a7-bbfb-48e630e8f3e3 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:47.204460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a450a9-bf38-47a7-bbfb-48e630e8f3e3 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:47.205846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:47.210715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafbfc5-1180-42bd-99c5-15938f82a8d6 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:47.211115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafbfc5-1180-42bd-99c5-15938f82a8d6 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:47.211268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafbfc5-1180-42bd-99c5-15938f82a8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:47.411827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-2bafbfc5-1180-42bd-99c5-15938f82a8d6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:47.412992 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2bafbfc5-1180-42bd-99c5-15938f82a8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:47.439657 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:47.508028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:47.663595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a450a9-bf38-47a7-bbfb-48e630e8f3e3 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:47.665102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-66a450a9-bf38-47a7-bbfb-48e630e8f3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:47.809651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a450a9-bf38-47a7-bbfb-48e630e8f3e3 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:47.850988 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:47.923349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c1484-cf97-4a58-89b7-7760d369cca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:47.923867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c1484-cf97-4a58-89b7-7760d369cca6 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:47.924168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c1484-cf97-4a58-89b7-7760d369cca6 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:47.924436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c1484-cf97-4a58-89b7-7760d369cca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:47.926051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-068c1484-cf97-4a58-89b7-7760d369cca6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:48.018347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c1484-cf97-4a58-89b7-7760d369cca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:48.019939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c1484-cf97-4a58-89b7-7760d369cca6 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:48.038095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 707/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:48.921122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3191bb2-c527-42f1-bd23-14f4324dc376 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:48.921674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3191bb2-c527-42f1-bd23-14f4324dc376 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:48.921674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3191bb2-c527-42f1-bd23-14f4324dc376 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:48.923035 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:23:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:48.928001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79411-af5a-4514-9b36-81c97f618d82 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:48.928459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79411-af5a-4514-9b36-81c97f618d82 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:48.928785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79411-af5a-4514-9b36-81c97f618d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:48.985644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:49.012368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-4af79411-af5a-4514-9b36-81c97f618d82 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:49.013621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4af79411-af5a-4514-9b36-81c97f618d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:49.112376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:23:49.263320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3191bb2-c527-42f1-bd23-14f4324dc376 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:49.264971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e3191bb2-c527-42f1-bd23-14f4324dc376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:49.408271 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3191bb2-c527-42f1-bd23-14f4324dc376 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:49.420690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:49.521006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd26a1-b4b8-49aa-a5c7-311c45d00266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:49.521606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd26a1-b4b8-49aa-a5c7-311c45d00266 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:49.521876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd26a1-b4b8-49aa-a5c7-311c45d00266 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:49.522140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd26a1-b4b8-49aa-a5c7-311c45d00266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:49.523761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3ddd26a1-b4b8-49aa-a5c7-311c45d00266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:49.611889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd26a1-b4b8-49aa-a5c7-311c45d00266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:49.613559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd26a1-b4b8-49aa-a5c7-311c45d00266 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:49.631801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 714/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:50.490880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b3e6b-6c92-45bf-954c-4b369620300e None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:50.491416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b3e6b-6c92-45bf-954c-4b369620300e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:50.491416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b3e6b-6c92-45bf-954c-4b369620300e None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:50.492769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:50.497655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda928e5-6a7e-4e35-a8f8-dd3ad09153f1 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:50.498059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda928e5-6a7e-4e35-a8f8-dd3ad09153f1 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:50.498868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda928e5-6a7e-4e35-a8f8-dd3ad09153f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:50.518416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:50.573879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bda928e5-6a7e-4e35-a8f8-dd3ad09153f1 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:50.575076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bda928e5-6a7e-4e35-a8f8-dd3ad09153f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:50.668361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:50.818961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b3e6b-6c92-45bf-954c-4b369620300e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:50.820555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8b9b3e6b-6c92-45bf-954c-4b369620300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:50.929408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:50.965485 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b3e6b-6c92-45bf-954c-4b369620300e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:51.078558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81595f2-cfa7-4022-bea8-83a522a113b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:51.079101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81595f2-cfa7-4022-bea8-83a522a113b7 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:51.079374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81595f2-cfa7-4022-bea8-83a522a113b7 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:51.079637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81595f2-cfa7-4022-bea8-83a522a113b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:51.081267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f81595f2-cfa7-4022-bea8-83a522a113b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:51.156683 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:51.170573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81595f2-cfa7-4022-bea8-83a522a113b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:51.172163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81595f2-cfa7-4022-bea8-83a522a113b7 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:51.190383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 710/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:52.078716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299246a8-0ba0-46a5-8d33-1ff6e78508b6 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:52.079288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299246a8-0ba0-46a5-8d33-1ff6e78508b6 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:52.079288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299246a8-0ba0-46a5-8d33-1ff6e78508b6 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:52.080636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:23:52.085496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757cc1f-b55b-4dfc-9e43-51cbdc7ce37f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:52.085939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757cc1f-b55b-4dfc-9e43-51cbdc7ce37f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:52.086050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757cc1f-b55b-4dfc-9e43-51cbdc7ce37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:52.162989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e757cc1f-b55b-4dfc-9e43-51cbdc7ce37f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:52.164989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e757cc1f-b55b-4dfc-9e43-51cbdc7ce37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:52.261504 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:52.412551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299246a8-0ba0-46a5-8d33-1ff6e78508b6 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:52.414073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-299246a8-0ba0-46a5-8d33-1ff6e78508b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:52.496086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:52.558003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299246a8-0ba0-46a5-8d33-1ff6e78508b6 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:52.668958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbfdbf-3258-4216-9f6f-6b4ca5c2866f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:52.669512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbfdbf-3258-4216-9f6f-6b4ca5c2866f service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:52.669787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbfdbf-3258-4216-9f6f-6b4ca5c2866f service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:52.670677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbfdbf-3258-4216-9f6f-6b4ca5c2866f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:52.672399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2ffbfdbf-3258-4216-9f6f-6b4ca5c2866f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:52.722380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:52.765335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbfdbf-3258-4216-9f6f-6b4ca5c2866f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:52.766948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffbfdbf-3258-4216-9f6f-6b4ca5c2866f service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:52.784744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 717/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:53.587666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22663137-567f-4fee-83ac-420e56d7dffb None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:53.588232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22663137-567f-4fee-83ac-420e56d7dffb None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:53.588232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22663137-567f-4fee-83ac-420e56d7dffb None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:53.589622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:53.595477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d88c-b5bf-475f-a37c-230fefe9b18e None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:53.595942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d88c-b5bf-475f-a37c-230fefe9b18e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:53.596382 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d88c-b5bf-475f-a37c-230fefe9b18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:53.673240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5f68d88c-b5bf-475f-a37c-230fefe9b18e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:53.674447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5f68d88c-b5bf-475f-a37c-230fefe9b18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:53.769490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:53.921090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22663137-567f-4fee-83ac-420e56d7dffb None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:53.922657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-22663137-567f-4fee-83ac-420e56d7dffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:53.972460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:54.068261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22663137-567f-4fee-83ac-420e56d7dffb None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:54.179452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b828b0f-0630-42c4-948b-ddd5bf26314d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:54.179974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b828b0f-0630-42c4-948b-ddd5bf26314d service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:54.180247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b828b0f-0630-42c4-948b-ddd5bf26314d service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:54.180534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b828b0f-0630-42c4-948b-ddd5bf26314d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:54.182122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0b828b0f-0630-42c4-948b-ddd5bf26314d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:54.189485 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:54.274984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b828b0f-0630-42c4-948b-ddd5bf26314d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:54.276621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b828b0f-0630-42c4-948b-ddd5bf26314d service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:54.294600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 713/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:55.095432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931f016-0e1e-40b9-ac78-51ae76f9f4f0 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:55.095920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931f016-0e1e-40b9-ac78-51ae76f9f4f0 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:55.096027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931f016-0e1e-40b9-ac78-51ae76f9f4f0 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:55.097374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:55.102150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4f50e-4b22-4e00-a9e4-d817725a1d24 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:55.102579 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4f50e-4b22-4e00-a9e4-d817725a1d24 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:55.102722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4f50e-4b22-4e00-a9e4-d817725a1d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:55.178645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-08e4f50e-4b22-4e00-a9e4-d817725a1d24 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:55.179864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-08e4f50e-4b22-4e00-a9e4-d817725a1d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:55.253667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:55.286376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:55.437609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931f016-0e1e-40b9-ac78-51ae76f9f4f0 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:55.439128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f931f016-0e1e-40b9-ac78-51ae76f9f4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:55.456545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:55.585725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931f016-0e1e-40b9-ac78-51ae76f9f4f0 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:55.681311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:55.707930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0840ff0-855c-454b-885b-ca615494e8be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:55.708601 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0840ff0-855c-454b-885b-ca615494e8be service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:55.708880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0840ff0-855c-454b-885b-ca615494e8be service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:55.709150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0840ff0-855c-454b-885b-ca615494e8be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:55.710767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b0840ff0-855c-454b-885b-ca615494e8be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:55.799695 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0840ff0-855c-454b-885b-ca615494e8be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:55.801600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0840ff0-855c-454b-885b-ca615494e8be service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:55.819209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 720/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:56.623391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a172cd-07b6-4e5e-93f4-aa8de26f2947 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:56.623961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a172cd-07b6-4e5e-93f4-aa8de26f2947 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:56.623961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a172cd-07b6-4e5e-93f4-aa8de26f2947 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:56.625295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:56.630106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd53998-d99e-49a2-9de4-1813e4972059 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:56.630548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd53998-d99e-49a2-9de4-1813e4972059 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:56.630645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd53998-d99e-49a2-9de4-1813e4972059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:56.705859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-dfd53998-d99e-49a2-9de4-1813e4972059 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:56.707073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-dfd53998-d99e-49a2-9de4-1813e4972059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:56.748881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:56.804808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:56.967395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a172cd-07b6-4e5e-93f4-aa8de26f2947 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:56.969073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-33a172cd-07b6-4e5e-93f4-aa8de26f2947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:57.131417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a172cd-07b6-4e5e-93f4-aa8de26f2947 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:57.194007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:57.263991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a047d64-25b5-4347-b554-db20d7034d19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:57.264839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a047d64-25b5-4347-b554-db20d7034d19 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:57.265116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a047d64-25b5-4347-b554-db20d7034d19 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:57.265383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a047d64-25b5-4347-b554-db20d7034d19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:57.267007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2a047d64-25b5-4347-b554-db20d7034d19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:57.363908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a047d64-25b5-4347-b554-db20d7034d19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:57.366251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a047d64-25b5-4347-b554-db20d7034d19 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:57.388622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 716/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:23:58.243491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55bfe7-172b-4d64-ac72-1b4150d8a9b8 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:58.243969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55bfe7-172b-4d64-ac72-1b4150d8a9b8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:58.244065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55bfe7-172b-4d64-ac72-1b4150d8a9b8 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:58.245443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:23:58.251089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02a658-8dea-437a-8b26-b82cc56e4ae5 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:58.251550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02a658-8dea-437a-8b26-b82cc56e4ae5 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:58.251666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02a658-8dea-437a-8b26-b82cc56e4ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:58.326939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2d02a658-8dea-437a-8b26-b82cc56e4ae5 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:23:58.328125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2d02a658-8dea-437a-8b26-b82cc56e4ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:58.330199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:58.432138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:23:58.583083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55bfe7-172b-4d64-ac72-1b4150d8a9b8 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:23:58.584653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4c55bfe7-172b-4d64-ac72-1b4150d8a9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:58.729388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55bfe7-172b-4d64-ac72-1b4150d8a9b8 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:23:58.755328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:23:58.845986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab9aa6-ccd0-4a70-b906-eef7f538ef77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:23:58.846501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab9aa6-ccd0-4a70-b906-eef7f538ef77 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:23:58.846775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab9aa6-ccd0-4a70-b906-eef7f538ef77 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:23:58.847040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab9aa6-ccd0-4a70-b906-eef7f538ef77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:23:58.848684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-40ab9aa6-ccd0-4a70-b906-eef7f538ef77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:23:58.936791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab9aa6-ccd0-4a70-b906-eef7f538ef77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:23:58.938398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab9aa6-ccd0-4a70-b906-eef7f538ef77 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:23:58.956063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 723/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:00.783208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17565d-8970-4a5a-8d3b-8c1a5e521172 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:00.783797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17565d-8970-4a5a-8d3b-8c1a5e521172 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:00.783797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17565d-8970-4a5a-8d3b-8c1a5e521172 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:00.785160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:00.790666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b041d-e18b-4997-960f-38c5e6c9a583 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:00.791081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b041d-e18b-4997-960f-38c5e6c9a583 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:00.791214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b041d-e18b-4997-960f-38c5e6c9a583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:00.832304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:00.858563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d84b041d-e18b-4997-960f-38c5e6c9a583 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:00.859797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d84b041d-e18b-4997-960f-38c5e6c9a583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:00.956439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:01.243722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17565d-8970-4a5a-8d3b-8c1a5e521172 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:01.245355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0b17565d-8970-4a5a-8d3b-8c1a5e521172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:01.350514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17565d-8970-4a5a-8d3b-8c1a5e521172 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:01.382598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:01.429778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f8bad-c738-4f6b-9d98-6e658b434bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:01.430381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f8bad-c738-4f6b-9d98-6e658b434bca service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:01.430651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f8bad-c738-4f6b-9d98-6e658b434bca service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:01.430917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f8bad-c738-4f6b-9d98-6e658b434bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:01.432527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0b5f8bad-c738-4f6b-9d98-6e658b434bca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:01.543249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f8bad-c738-4f6b-9d98-6e658b434bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:01.545793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f8bad-c738-4f6b-9d98-6e658b434bca service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:01.565191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 719/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:03.642943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7956c-cd2d-41cb-a1f4-771b64867e21 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:03.643522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7956c-cd2d-41cb-a1f4-771b64867e21 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:03.643522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7956c-cd2d-41cb-a1f4-771b64867e21 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:03.644896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:03.650194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d9b77-88aa-40c5-b629-cc13dba2cfc7 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:03.650654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d9b77-88aa-40c5-b629-cc13dba2cfc7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:03.650763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d9b77-88aa-40c5-b629-cc13dba2cfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:03.717468 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ff3d9b77-88aa-40c5-b629-cc13dba2cfc7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:03.718672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ff3d9b77-88aa-40c5-b629-cc13dba2cfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:03.731611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:03.817809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:04.105583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7956c-cd2d-41cb-a1f4-771b64867e21 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:04.107653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-64d7956c-cd2d-41cb-a1f4-771b64867e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:04.210231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7956c-cd2d-41cb-a1f4-771b64867e21 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:04.223155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:04.284851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce0721-bd9d-49a3-9edc-5057306013d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:04.285455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce0721-bd9d-49a3-9edc-5057306013d4 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:04.285731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce0721-bd9d-49a3-9edc-5057306013d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:04.286000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce0721-bd9d-49a3-9edc-5057306013d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:04.287607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0cce0721-bd9d-49a3-9edc-5057306013d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:04.396098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce0721-bd9d-49a3-9edc-5057306013d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:04.397492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce0721-bd9d-49a3-9edc-5057306013d4 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:04.416265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 726/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:06.619224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727acae-1bab-496d-bca8-c25fcea2157c None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:06.619779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727acae-1bab-496d-bca8-c25fcea2157c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:06.619898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727acae-1bab-496d-bca8-c25fcea2157c None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:06.621247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:06.627085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c14a-9a17-450e-9bcc-07f2e5215ac5 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:06.627483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c14a-9a17-450e-9bcc-07f2e5215ac5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:06.627625 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120c14a-9a17-450e-9bcc-07f2e5215ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:06.674823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:06.693119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-4120c14a-9a17-450e-9bcc-07f2e5215ac5 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:06.694339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4120c14a-9a17-450e-9bcc-07f2e5215ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:06.792141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:07.074529 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727acae-1bab-496d-bca8-c25fcea2157c None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:07.076659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6727acae-1bab-496d-bca8-c25fcea2157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:07.180377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727acae-1bab-496d-bca8-c25fcea2157c None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:07.221310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:07.256983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6d2ee-9021-48eb-98bf-9c0574aa5660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:07.257592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6d2ee-9021-48eb-98bf-9c0574aa5660 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:07.257868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6d2ee-9021-48eb-98bf-9c0574aa5660 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:07.258136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6d2ee-9021-48eb-98bf-9c0574aa5660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:07.259727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4ce6d2ee-9021-48eb-98bf-9c0574aa5660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:07.368415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6d2ee-9021-48eb-98bf-9c0574aa5660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:07.369930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6d2ee-9021-48eb-98bf-9c0574aa5660 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:07.389223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 722/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:09.400888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9c62b-b204-462a-a5f1-7ef4d64ba158 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:09.401395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9c62b-b204-462a-a5f1-7ef4d64ba158 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:09.401395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9c62b-b204-462a-a5f1-7ef4d64ba158 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:09.402734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:09.408156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2cd41-8f22-4ee4-b6ec-ce4537ff70c6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:09.408633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2cd41-8f22-4ee4-b6ec-ce4537ff70c6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:09.408743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2cd41-8f22-4ee4-b6ec-ce4537ff70c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:09.474605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-97f2cd41-8f22-4ee4-b6ec-ce4537ff70c6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:09.475793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-97f2cd41-8f22-4ee4-b6ec-ce4537ff70c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:09.496409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:09.571609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:09.859154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9c62b-b204-462a-a5f1-7ef4d64ba158 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:09.860760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1ec9c62b-b204-462a-a5f1-7ef4d64ba158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:09.962732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9c62b-b204-462a-a5f1-7ef4d64ba158 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:09.984344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:10.037894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d47c9-f28c-440d-8281-b04c0b79c005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:10.038499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d47c9-f28c-440d-8281-b04c0b79c005 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:10.038775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d47c9-f28c-440d-8281-b04c0b79c005 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:10.039045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d47c9-f28c-440d-8281-b04c0b79c005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:10.040679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c56d47c9-f28c-440d-8281-b04c0b79c005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:10.146378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d47c9-f28c-440d-8281-b04c0b79c005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:10.147748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d47c9-f28c-440d-8281-b04c0b79c005 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:10.167375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 729/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:12.120001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69800f92-d8aa-4d19-939a-2f5737a6cf5b None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:12.120610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69800f92-d8aa-4d19-939a-2f5737a6cf5b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:12.120610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69800f92-d8aa-4d19-939a-2f5737a6cf5b None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:12.121913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:24:12.127069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e6750-5325-4398-9a83-8e82e6c57e75 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:12.127462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e6750-5325-4398-9a83-8e82e6c57e75 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:12.127608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e6750-5325-4398-9a83-8e82e6c57e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:12.189407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:12.192288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-897e6750-5325-4398-9a83-8e82e6c57e75 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:12.194137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-897e6750-5325-4398-9a83-8e82e6c57e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:12.292914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:12.583801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69800f92-d8aa-4d19-939a-2f5737a6cf5b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:12.585379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-69800f92-d8aa-4d19-939a-2f5737a6cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:12.688655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69800f92-d8aa-4d19-939a-2f5737a6cf5b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:12.738065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:12.762519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ae19c-098f-401f-9bb6-78c4a0ce1763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:12.763187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ae19c-098f-401f-9bb6-78c4a0ce1763 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:12.763496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ae19c-098f-401f-9bb6-78c4a0ce1763 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:12.763768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ae19c-098f-401f-9bb6-78c4a0ce1763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:12.765413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d97ae19c-098f-401f-9bb6-78c4a0ce1763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:12.875626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ae19c-098f-401f-9bb6-78c4a0ce1763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:12.877041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ae19c-098f-401f-9bb6-78c4a0ce1763 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:12.896000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 725/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:14.872474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b584cbb-93da-4b7b-848a-20ab71d44aaa None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:14.872474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b584cbb-93da-4b7b-848a-20ab71d44aaa None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:14.873351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b584cbb-93da-4b7b-848a-20ab71d44aaa None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:14.874312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:14.879556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427933a-33ef-4b59-be3f-c2156344e015 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:14.880014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427933a-33ef-4b59-be3f-c2156344e015 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:14.880131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427933a-33ef-4b59-be3f-c2156344e015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:14.947795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1427933a-33ef-4b59-be3f-c2156344e015 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:14.949033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1427933a-33ef-4b59-be3f-c2156344e015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:14.978542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:15.046246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:15.332876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b584cbb-93da-4b7b-848a-20ab71d44aaa None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:15.334419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0b584cbb-93da-4b7b-848a-20ab71d44aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:15.438239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b584cbb-93da-4b7b-848a-20ab71d44aaa None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:15.466641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:15.513629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e64503-9a25-49c2-8458-e6c5d58e4e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:15.514237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e64503-9a25-49c2-8458-e6c5d58e4e31 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:15.514513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e64503-9a25-49c2-8458-e6c5d58e4e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:15.514785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e64503-9a25-49c2-8458-e6c5d58e4e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:15.516387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-99e64503-9a25-49c2-8458-e6c5d58e4e31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:15.624293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e64503-9a25-49c2-8458-e6c5d58e4e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:15.625769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e64503-9a25-49c2-8458-e6c5d58e4e31 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:15.645008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 732/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:17.601633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c6991-e811-433b-af44-453aa38f7d4c None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:17.602167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c6991-e811-433b-af44-453aa38f7d4c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:17.602167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c6991-e811-433b-af44-453aa38f7d4c None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:17.603599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:17.609307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08171805-224e-4a12-ad64-0c786169b14e None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:17.609755 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08171805-224e-4a12-ad64-0c786169b14e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:17.610147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08171805-224e-4a12-ad64-0c786169b14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:17.686137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-08171805-224e-4a12-ad64-0c786169b14e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:17.687328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-08171805-224e-4a12-ad64-0c786169b14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:17.695454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:17.784804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:18.080890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c6991-e811-433b-af44-453aa38f7d4c None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:18.082415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-984c6991-e811-433b-af44-453aa38f7d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:18.185907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c6991-e811-433b-af44-453aa38f7d4c None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:18.241180 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:18.263553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5677d-87be-4882-944d-7dd5e14f8b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:18.264164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5677d-87be-4882-944d-7dd5e14f8b18 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:18.264441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5677d-87be-4882-944d-7dd5e14f8b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:18.264738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5677d-87be-4882-944d-7dd5e14f8b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:18.266366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-def5677d-87be-4882-944d-7dd5e14f8b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:18.374775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5677d-87be-4882-944d-7dd5e14f8b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:18.376135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5677d-87be-4882-944d-7dd5e14f8b18 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:18.395777 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 728/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:20.366318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0a60b-b35a-4b6b-874b-18f9e23ac1e0 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:20.366883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0a60b-b35a-4b6b-874b-18f9e23ac1e0 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:20.366883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0a60b-b35a-4b6b-874b-18f9e23ac1e0 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:20.368203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:24:20.374260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2be71-c820-4842-b3c5-e12a96038636 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:20.374700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2be71-c820-4842-b3c5-e12a96038636 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:20.374818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2be71-c820-4842-b3c5-e12a96038636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:20.440375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1be2be71-c820-4842-b3c5-e12a96038636 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:20.441603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1be2be71-c820-4842-b3c5-e12a96038636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:20.480175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:20.538362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:20.818305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0a60b-b35a-4b6b-874b-18f9e23ac1e0 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:20.819834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-adc0a60b-b35a-4b6b-874b-18f9e23ac1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:20.924779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0a60b-b35a-4b6b-874b-18f9e23ac1e0 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:20.962859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:20.999917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009d5fb-ecf7-41fe-b4ee-e4038b569289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:21.000552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009d5fb-ecf7-41fe-b4ee-e4038b569289 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:21.000837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009d5fb-ecf7-41fe-b4ee-e4038b569289 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:21.001115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009d5fb-ecf7-41fe-b4ee-e4038b569289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:21.002786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3009d5fb-ecf7-41fe-b4ee-e4038b569289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:21.110528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009d5fb-ecf7-41fe-b4ee-e4038b569289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:21.111969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009d5fb-ecf7-41fe-b4ee-e4038b569289 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:21.130679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 735/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:23.036413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210d8a4-9e8e-4785-ae60-f30b198846d8 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:23.036977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210d8a4-9e8e-4785-ae60-f30b198846d8 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:23.037124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210d8a4-9e8e-4785-ae60-f30b198846d8 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:23.038740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:23.044615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e76ba-bddc-480c-a984-7154a7603697 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:23.045052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e76ba-bddc-480c-a984-7154a7603697 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:23.045190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e76ba-bddc-480c-a984-7154a7603697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:23.111648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bc2e76ba-bddc-480c-a984-7154a7603697 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:23.112865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bc2e76ba-bddc-480c-a984-7154a7603697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:23.130300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:23.207624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:23.491102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210d8a4-9e8e-4785-ae60-f30b198846d8 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:23.492682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2210d8a4-9e8e-4785-ae60-f30b198846d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:23.596276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210d8a4-9e8e-4785-ae60-f30b198846d8 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:23.662429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:23.669813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86c87d-9ddc-437e-8332-d8b926414d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:23.670416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86c87d-9ddc-437e-8332-d8b926414d52 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:23.670689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86c87d-9ddc-437e-8332-d8b926414d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:23.670958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86c87d-9ddc-437e-8332-d8b926414d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:23.672635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ea86c87d-9ddc-437e-8332-d8b926414d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:23.781614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea86c87d-9ddc-437e-8332-d8b926414d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:23.782996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea86c87d-9ddc-437e-8332-d8b926414d52 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:23.802552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 731/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:25.724702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf355932-b87e-4eb2-9eea-1340618d61ad None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:25.725264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf355932-b87e-4eb2-9eea-1340618d61ad None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:25.725264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf355932-b87e-4eb2-9eea-1340618d61ad None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:25.726617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:25.732297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b913f3-bd31-41cb-8efa-45df49fbe6de None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:25.732742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b913f3-bd31-41cb-8efa-45df49fbe6de None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:25.732878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b913f3-bd31-41cb-8efa-45df49fbe6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:25.798763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-c4b913f3-bd31-41cb-8efa-45df49fbe6de None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:25.800228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-c4b913f3-bd31-41cb-8efa-45df49fbe6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:25.848817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:25.897786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:26.181161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf355932-b87e-4eb2-9eea-1340618d61ad None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:26.182757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bf355932-b87e-4eb2-9eea-1340618d61ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:26.287088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf355932-b87e-4eb2-9eea-1340618d61ad None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:26.334967 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:26.361473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b001e2f-8ab3-4b3e-a0d5-c67459ac3ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:26.362071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001e2f-8ab3-4b3e-a0d5-c67459ac3ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:26.362349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001e2f-8ab3-4b3e-a0d5-c67459ac3ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:26.362619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001e2f-8ab3-4b3e-a0d5-c67459ac3ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:26.364217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8b001e2f-8ab3-4b3e-a0d5-c67459ac3ce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:26.470940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b001e2f-8ab3-4b3e-a0d5-c67459ac3ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:26.472303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b001e2f-8ab3-4b3e-a0d5-c67459ac3ce1 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:26.490887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 738/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:35.896011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=22 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:36.488701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=66 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:45.995553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a0949-4563-4cb8-ac67-57bd18284384 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:45.996118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a0949-4563-4cb8-ac67-57bd18284384 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:45.996118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a0949-4563-4cb8-ac67-57bd18284384 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:45.997502 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 733/1471] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:46.002729 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173d550-f8dd-4224-aeb8-f36f1b951a25 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:46.003116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173d550-f8dd-4224-aeb8-f36f1b951a25 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:46.003270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173d550-f8dd-4224-aeb8-f36f1b951a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:46.070169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e173d550-f8dd-4224-aeb8-f36f1b951a25 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:46.071373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e173d550-f8dd-4224-aeb8-f36f1b951a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:46.166688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 739/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:46.471559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468e21-458f-4b71-b3fa-9dc80600a51b None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:46.471840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468e21-458f-4b71-b3fa-9dc80600a51b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:46.472109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03468e21-458f-4b71-b3fa-9dc80600a51b None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:46.473477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 734/1473] 10.222.0.25 () {54 vars in 967 bytes} [Thu Sep 1 00:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:46.478440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d991c-e46f-4c02-adf1-a41afac2924c None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:46.478726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d991c-e46f-4c02-adf1-a41afac2924c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:46.478999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d991c-e46f-4c02-adf1-a41afac2924c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:46.553914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-615d991c-e46f-4c02-adf1-a41afac2924c None None] MFA Rules not processed for user `52470b620abd4cd29ab7e4752f285bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:46.555114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-615d991c-e46f-4c02-adf1-a41afac2924c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:46.575852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:46.630735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 740/1474] 10.222.0.25 () {58 vars in 1064 bytes} [Thu Sep 1 00:24:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:46.638322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03468e21-458f-4b71-b3fa-9dc80600a51b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:46.639854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-03468e21-458f-4b71-b3fa-9dc80600a51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:46.742781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03468e21-458f-4b71-b3fa-9dc80600a51b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:46.816419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d2bd9-b4cc-4a47-92b7-ce766cf98c08 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52470b620abd4cd29ab7e4752f285bb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:46.817047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d2bd9-b4cc-4a47-92b7-ce766cf98c08 service glance] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:46.817321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d2bd9-b4cc-4a47-92b7-ce766cf98c08 service glance] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:46.817593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d2bd9-b4cc-4a47-92b7-ce766cf98c08 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:46.819192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-585d2bd9-b4cc-4a47-92b7-ce766cf98c08 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:46.875801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:46.930870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d2bd9-b4cc-4a47-92b7-ce766cf98c08 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:46.932309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d2bd9-b4cc-4a47-92b7-ce766cf98c08 service glance] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:46.951433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 735/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:48.929331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfc75e-be6d-44d7-86ec-fd4a2ae8dfec None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:48.929932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfc75e-be6d-44d7-86ec-fd4a2ae8dfec None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:48.929932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfc75e-be6d-44d7-86ec-fd4a2ae8dfec None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:48.931221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 741/1476] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:48.937242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391985d8-562c-427c-8169-1006cf1cca40 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:48.937674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391985d8-562c-427c-8169-1006cf1cca40 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:48.937810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391985d8-562c-427c-8169-1006cf1cca40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:49.004126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-391985d8-562c-427c-8169-1006cf1cca40 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:49.005346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-391985d8-562c-427c-8169-1006cf1cca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:49.105852 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 736/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:49.375513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfc75e-be6d-44d7-86ec-fd4a2ae8dfec None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:49.377110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a1bfc75e-be6d-44d7-86ec-fd4a2ae8dfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:49.506810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfc75e-be6d-44d7-86ec-fd4a2ae8dfec None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:49.557303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:49.602112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d049a2-fdc7-4c79-a2dd-3b34da88e36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:49.602707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d049a2-fdc7-4c79-a2dd-3b34da88e36c admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:49.602979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d049a2-fdc7-4c79-a2dd-3b34da88e36c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:49.603244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d049a2-fdc7-4c79-a2dd-3b34da88e36c admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:49.603872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d049a2-fdc7-4c79-a2dd-3b34da88e36c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:49.606324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING py.warnings [None req-66d049a2-fdc7-4c79-a2dd-3b34da88e36c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:24:49.606324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: warnings.warn(msg) Sep 01 00:24:49.606324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: Sep 01 00:24:49.607461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d049a2-fdc7-4c79-a2dd-3b34da88e36c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:49.627323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 742/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 1 00:24:49 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:24:51.551291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7248e4-3980-4e15-83b7-40e28719e336 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:51.551912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7248e4-3980-4e15-83b7-40e28719e336 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:51.551912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7248e4-3980-4e15-83b7-40e28719e336 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:51.553230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 737/1479] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:51.558968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3e7be-15d3-4292-be27-bf43291b83c8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:51.559410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3e7be-15d3-4292-be27-bf43291b83c8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:51.559518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3e7be-15d3-4292-be27-bf43291b83c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:51.624907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a8c3e7be-15d3-4292-be27-bf43291b83c8 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:51.626119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a8c3e7be-15d3-4292-be27-bf43291b83c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:51.721291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 743/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:52.021904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7248e4-3980-4e15-83b7-40e28719e336 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:52.023477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ec7248e4-3980-4e15-83b7-40e28719e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:52.033519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:52.169188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7248e4-3980-4e15-83b7-40e28719e336 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:52.249560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:52.283318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b770125-5143-4472-80df-d4fcfbeb5ad7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:52.283841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b770125-5143-4472-80df-d4fcfbeb5ad7 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:52.284115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b770125-5143-4472-80df-d4fcfbeb5ad7 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:52.284383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b770125-5143-4472-80df-d4fcfbeb5ad7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:52.286014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0b770125-5143-4472-80df-d4fcfbeb5ad7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:52.375654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b770125-5143-4472-80df-d4fcfbeb5ad7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:52.377317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b770125-5143-4472-80df-d4fcfbeb5ad7 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:52.395738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 738/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:54.480364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b71839-7935-4b87-81ae-8f6460cd9787 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:54.480959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b71839-7935-4b87-81ae-8f6460cd9787 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:54.480959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b71839-7935-4b87-81ae-8f6460cd9787 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:54.482287 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 744/1482] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:54.487619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487162d-8e0b-4924-ac2a-c5c0eb3855a5 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:54.488019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487162d-8e0b-4924-ac2a-c5c0eb3855a5 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:54.488147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487162d-8e0b-4924-ac2a-c5c0eb3855a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:54.554856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-4487162d-8e0b-4924-ac2a-c5c0eb3855a5 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:54.556052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4487162d-8e0b-4924-ac2a-c5c0eb3855a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:54.604881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:54.656449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 739/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:54.945082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b71839-7935-4b87-81ae-8f6460cd9787 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:54.946671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a3b71839-7935-4b87-81ae-8f6460cd9787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:54.971501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:55.050158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b71839-7935-4b87-81ae-8f6460cd9787 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:55.122170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4c155-2470-45a0-a248-ab487f2272be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:55.122766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4c155-2470-45a0-a248-ab487f2272be service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:55.123042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4c155-2470-45a0-a248-ab487f2272be service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:55.123311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4c155-2470-45a0-a248-ab487f2272be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:55.124935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-04a4c155-2470-45a0-a248-ab487f2272be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:55.200803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:55.237681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4c155-2470-45a0-a248-ab487f2272be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:55.239142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4c155-2470-45a0-a248-ab487f2272be service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:55.258374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 745/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:24:57.211116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180a667-59ec-4bc9-ab59-f9bda99dee6f None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:57.211659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180a667-59ec-4bc9-ab59-f9bda99dee6f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:57.211659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180a667-59ec-4bc9-ab59-f9bda99dee6f None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:57.213064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 740/1485] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:24:57.219190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef33678-69a7-4d10-8862-c12ba603b039 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:57.219609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef33678-69a7-4d10-8862-c12ba603b039 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:57.219730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef33678-69a7-4d10-8862-c12ba603b039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:57.287122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-1ef33678-69a7-4d10-8862-c12ba603b039 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:24:57.288330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1ef33678-69a7-4d10-8862-c12ba603b039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:57.383398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 746/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:24:57.674143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180a667-59ec-4bc9-ab59-f9bda99dee6f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:24:57.675695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e180a667-59ec-4bc9-ab59-f9bda99dee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:57.784799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180a667-59ec-4bc9-ab59-f9bda99dee6f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:24:57.867181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:24:57.869118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcd4ef-cd51-4e79-ae22-3e51ee91b4e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:24:57.870100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcd4ef-cd51-4e79-ae22-3e51ee91b4e7 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:24:57.871066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcd4ef-cd51-4e79-ae22-3e51ee91b4e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:24:57.871385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcd4ef-cd51-4e79-ae22-3e51ee91b4e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:24:57.873348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d4bcd4ef-cd51-4e79-ae22-3e51ee91b4e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:24:57.987562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcd4ef-cd51-4e79-ae22-3e51ee91b4e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:24:57.989921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcd4ef-cd51-4e79-ae22-3e51ee91b4e7 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:24:58.014462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 741/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:25:00.072038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c55994-d9db-4567-b246-757d08bc6e5c None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:25:00.073137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c55994-d9db-4567-b246-757d08bc6e5c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:25:00.073137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c55994-d9db-4567-b246-757d08bc6e5c None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:25:00.074594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 747/1488] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:25:00.079691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03900c64-7866-440f-bcbc-8162aae7ed55 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:25:00.080125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03900c64-7866-440f-bcbc-8162aae7ed55 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:25:00.080267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03900c64-7866-440f-bcbc-8162aae7ed55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:25:00.146027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-03900c64-7866-440f-bcbc-8162aae7ed55 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:25:00.147236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-03900c64-7866-440f-bcbc-8162aae7ed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:25:00.199672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:25:00.246093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 742/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:25:00.538569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c55994-d9db-4567-b246-757d08bc6e5c None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:25:00.540096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f8c55994-d9db-4567-b246-757d08bc6e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:25:00.576068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:25:00.642931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c55994-d9db-4567-b246-757d08bc6e5c None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:25:00.715414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24758bf-d5e9-4017-8f85-00edefb965a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:25:00.716016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24758bf-d5e9-4017-8f85-00edefb965a2 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:25:00.716289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24758bf-d5e9-4017-8f85-00edefb965a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:25:00.716587 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24758bf-d5e9-4017-8f85-00edefb965a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:25:00.718181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c24758bf-d5e9-4017-8f85-00edefb965a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:25:00.796554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:25:00.827117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24758bf-d5e9-4017-8f85-00edefb965a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:25:00.828602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24758bf-d5e9-4017-8f85-00edefb965a2 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:25:00.847728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 748/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:25:02.927803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38efbd-80a9-47e2-80d0-c86cd3499ec0 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:25:02.928347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38efbd-80a9-47e2-80d0-c86cd3499ec0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:25:02.928347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38efbd-80a9-47e2-80d0-c86cd3499ec0 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:25:02.929751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 743/1491] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 1 00:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:25:02.935385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05318558-e80d-4b24-9ec7-0cc39d448a6a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:25:02.935846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05318558-e80d-4b24-9ec7-0cc39d448a6a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:25:02.935950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05318558-e80d-4b24-9ec7-0cc39d448a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:25:03.007349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-05318558-e80d-4b24-9ec7-0cc39d448a6a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:25:03.008566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-05318558-e80d-4b24-9ec7-0cc39d448a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:25:03.105451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 749/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 1 00:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:25:03.397213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38efbd-80a9-47e2-80d0-c86cd3499ec0 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:25:03.398786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3e38efbd-80a9-47e2-80d0-c86cd3499ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:25:03.502345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38efbd-80a9-47e2-80d0-c86cd3499ec0 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:25:03.576268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb263441-8a91-4d6d-891d-8048dda85c4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:25:03.576904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb263441-8a91-4d6d-891d-8048dda85c4a service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:25:03.577181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb263441-8a91-4d6d-891d-8048dda85c4a service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:25:03.577451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb263441-8a91-4d6d-891d-8048dda85c4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:25:03.579058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-bb263441-8a91-4d6d-891d-8048dda85c4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:25:03.588188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:25:03.687606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb263441-8a91-4d6d-891d-8048dda85c4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:25:03.689082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb263441-8a91-4d6d-891d-8048dda85c4a service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:25:03.706850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 744/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:25:13.103142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=82 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:25:13.704507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=50 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:19.617836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d65a8-f9e6-4db4-a60c-ecef3d8ba856 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:19.618415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d65a8-f9e6-4db4-a60c-ecef3d8ba856 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:19.618415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d65a8-f9e6-4db4-a60c-ecef3d8ba856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:19.694280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-872d65a8-f9e6-4db4-a60c-ecef3d8ba856 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:26:19.695500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-872d65a8-f9e6-4db4-a60c-ecef3d8ba856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:19.800354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 750/1494] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:26:19.809194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54c028-b638-48fd-aab3-308168d79767 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:19.809752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54c028-b638-48fd-aab3-308168d79767 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:19.809886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54c028-b638-48fd-aab3-308168d79767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:19.886875 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-dd54c028-b638-48fd-aab3-308168d79767 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:26:19.887976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-dd54c028-b638-48fd-aab3-308168d79767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:19.987633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 745/1495] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:26:19.992419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d65a8-f9e6-4db4-a60c-ecef3d8ba856 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:19.993661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-872d65a8-f9e6-4db4-a60c-ecef3d8ba856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:20.004816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d65a8-f9e6-4db4-a60c-ecef3d8ba856 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:20.074413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:20.108717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:20.109316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:20.109588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:20.109853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:20.110484 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:26:20.110780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:20.111796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING py.warnings [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:26:20.111796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: warnings.warn(msg) Sep 01 00:26:20.111796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: Sep 01 00:26:20.112697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:20.123939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 751/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:26:20.132454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54c028-b638-48fd-aab3-308168d79767 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:20.141784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54c028-b638-48fd-aab3-308168d79767 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:20.204016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:20.238441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:20.239048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:20.239318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:20.239578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:20.240223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:20.242152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WARNING py.warnings [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:26:20.242152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: warnings.warn(msg) Sep 01 00:26:20.242152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: Sep 01 00:26:20.243349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:20.303039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 746/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:26:20 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:26:20.307129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:20.308693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:20.347603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba72d3ee-b716-41c4-8d03-0c4c0b7b0e82 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:20.447555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:20.448071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:20.448342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:20.448642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:20.449259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:20.450271 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING py.warnings [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:26:20.450271 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: warnings.warn(msg) Sep 01 00:26:20.450271 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: Sep 01 00:26:20.451481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:20.493794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 752/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:26:20 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:26:20.498063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:20.499612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:20.561731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7aa5c-c9a2-40cc-b178-a531bf6c32c5 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:20.644596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:20.656233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:20.656867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:20.657144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:20.657442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:20.658075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:20.659088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WARNING py.warnings [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:26:20.659088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: warnings.warn(msg) Sep 01 00:26:20.659088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: Sep 01 00:26:20.660281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:20.681034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 747/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:26:20.685103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:20.686666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:20.698138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02754888-e2f0-46fb-beea-34d64f08e3f1 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:20.723600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:20.793629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84479cf3-e1e6-407f-a7e1-a3dec913bfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:20.794231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84479cf3-e1e6-407f-a7e1-a3dec913bfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:20.794500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84479cf3-e1e6-407f-a7e1-a3dec913bfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:20.794763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84479cf3-e1e6-407f-a7e1-a3dec913bfc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:20.795391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84479cf3-e1e6-407f-a7e1-a3dec913bfc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:20.796409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING py.warnings [None req-84479cf3-e1e6-407f-a7e1-a3dec913bfc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:26:20.796409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: warnings.warn(msg) Sep 01 00:26:20.796409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: Sep 01 00:26:20.797326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84479cf3-e1e6-407f-a7e1-a3dec913bfc1 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:20.809672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 753/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:26:20.814121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:20.815622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:20.849000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4eee7-c3a2-418d-8b98-fd416d0cc1d7 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:20.942545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c579f-61fe-428a-b874-69792cdbc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:20.943213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c579f-61fe-428a-b874-69792cdbc869 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:20.943554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c579f-61fe-428a-b874-69792cdbc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:20.943943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c579f-61fe-428a-b874-69792cdbc869 admin admin] PATH_INFO: `/v3/projects/5d9123ad7e5244d3b09a9d9dab4f7a95/users/f864dde9c313401fbfdfc54d6814c41e/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:20.967425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c579f-61fe-428a-b874-69792cdbc869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9123ad7e5244d3b09a9d9dab4f7a95, user_id=f864dde9c313401fbfdfc54d6814c41e, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:20.968390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WARNING py.warnings [None req-7c6c579f-61fe-428a-b874-69792cdbc869 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:26:20.968390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: warnings.warn(msg) Sep 01 00:26:20.968390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: Sep 01 00:26:20.970527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c579f-61fe-428a-b874-69792cdbc869 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:20.995892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 748/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:26:20 2022] PUT /identity/v3/projects/5d9123ad7e5244d3b09a9d9dab4f7a95/users/f864dde9c313401fbfdfc54d6814c41e/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:26:21.008001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48a55d-ea39-433f-a7eb-09ba7d96993f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:21.008289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48a55d-ea39-433f-a7eb-09ba7d96993f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:21.008600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48a55d-ea39-433f-a7eb-09ba7d96993f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:21.064692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-2c48a55d-ea39-433f-a7eb-09ba7d96993f None None] MFA Rules not processed for user `f864dde9c313401fbfdfc54d6814c41e`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:26:21.065901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2c48a55d-ea39-433f-a7eb-09ba7d96993f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:21.141996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 754/1502] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:26:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:26:21.155844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220dd30-2341-4354-9529-7fea6233009a None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:21.156152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220dd30-2341-4354-9529-7fea6233009a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:21.156426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220dd30-2341-4354-9529-7fea6233009a None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:21.157808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 749/1503] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 1 00:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:26:21.162762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9721adb-0bc6-460d-a47e-523298fcb214 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:21.163052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9721adb-0bc6-460d-a47e-523298fcb214 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:21.163328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9721adb-0bc6-460d-a47e-523298fcb214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:21.195044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:21.240181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f9721adb-0bc6-460d-a47e-523298fcb214 None None] MFA Rules not processed for user `bb5590d8961d47aa846117a236730d93`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:26:21.241434 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f9721adb-0bc6-460d-a47e-523298fcb214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:21.319936 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 755/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 1 00:26:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:26:21.329678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220dd30-2341-4354-9529-7fea6233009a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:21.331265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9220dd30-2341-4354-9529-7fea6233009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:21.382660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:21.439750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220dd30-2341-4354-9529-7fea6233009a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:21.512827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:21.513358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:21.513632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:21.513900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:21.515488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:21.616508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:21.618392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:21.621864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:21.637981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 750/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:26:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:26:21.682381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22f3ad-6304-4880-8a32-9a371f3b9bd4 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:21.682668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22f3ad-6304-4880-8a32-9a371f3b9bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:21.682989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22f3ad-6304-4880-8a32-9a371f3b9bd4 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:21.684284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 756/1506] 10.222.0.25 () {52 vars in 830 bytes} [Thu Sep 1 00:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:26:21.687916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:21.689156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:21.788221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0853087-2388-4ef8-a6a0-1fd40eb58471 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:21.881455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:21.882097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:21.882331 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:21.882594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] PATH_INFO: `/v3/users/f864dde9c313401fbfdfc54d6814c41e` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:21.894490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f864dde9c313401fbfdfc54d6814c41e)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:21.895513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WARNING py.warnings [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:26:21.895513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: warnings.warn(msg) Sep 01 00:26:21.895513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: Sep 01 00:26:21.896771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:21.906227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:21.963613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f864dde9c313401fbfdfc54d6814c41e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:21.964144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f864dde9c313401fbfdfc54d6814c41e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:21.988299 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f864dde9c313401fbfdfc54d6814c41e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:22.002471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f864dde9c313401fbfdfc54d6814c41e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:22.011679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f864dde9c313401fbfdfc54d6814c41e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:22.017415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 751/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:26:21 2022] DELETE /identity/v3/users/f864dde9c313401fbfdfc54d6814c41e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:26:22.022034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0a173-5992-4da8-ba15-1945b4449c43 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:22.022519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0a173-5992-4da8-ba15-1945b4449c43 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:22.022636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0a173-5992-4da8-ba15-1945b4449c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:22.079398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-27e0a173-5992-4da8-ba15-1945b4449c43 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:26:22.080557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-27e0a173-5992-4da8-ba15-1945b4449c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:22.155528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:22.169520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 757/1508] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:26:22.179519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:22.180768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:22.281917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390703f-b5e6-4c52-9a13-036a4a52cc04 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:22.335774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:22.356379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fc751-43fb-437e-bcd4-caedd5780853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:22.357016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fc751-43fb-437e-bcd4-caedd5780853 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:22.357289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fc751-43fb-437e-bcd4-caedd5780853 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:22.357556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fc751-43fb-437e-bcd4-caedd5780853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:22.359147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4d2fc751-43fb-437e-bcd4-caedd5780853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:22.466641 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2fc751-43fb-437e-bcd4-caedd5780853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:22.468082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2fc751-43fb-437e-bcd4-caedd5780853 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:22.485942 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 752/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:26:22.813403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0a173-5992-4da8-ba15-1945b4449c43 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:26:22.814869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-27e0a173-5992-4da8-ba15-1945b4449c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:26:22.826425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0a173-5992-4da8-ba15-1945b4449c43 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:26:22.923109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:26:22.923658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:26:22.924025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:26:22.924293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] PATH_INFO: `/v3/projects/5d9123ad7e5244d3b09a9d9dab4f7a95` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:26:22.934968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9123ad7e5244d3b09a9d9dab4f7a95)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:26:22.936010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING py.warnings [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:26:22.936010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: warnings.warn(msg) Sep 01 00:26:22.936010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: Sep 01 00:26:22.937312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:26:22.996222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9123ad7e5244d3b09a9d9dab4f7a95', 'request_id': 'req-21dc874d-b6f8-4d3f-b107-5801cf606f85'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:23.002347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9123ad7e5244d3b09a9d9dab4f7a95', 'request_id': 'req-21dc874d-b6f8-4d3f-b107-5801cf606f85'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:23.015233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9123ad7e5244d3b09a9d9dab4f7a95', 'request_id': 'req-21dc874d-b6f8-4d3f-b107-5801cf606f85'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:23.042156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] The token cache is being invalidate because project 5d9123ad7e5244d3b09a9d9dab4f7a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:26:23.042797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:23.042937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-21dc874d-b6f8-4d3f-b107-5801cf606f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:26:23.044232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 758/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:26:22 2022] DELETE /identity/v3/projects/5d9123ad7e5244d3b09a9d9dab4f7a95 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:26:32.483802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=62 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:26:33.040204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:27:47.837589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90063cb3-5356-4d22-b070-c06775a33a3b None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:47.838195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90063cb3-5356-4d22-b070-c06775a33a3b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:47.838195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90063cb3-5356-4d22-b070-c06775a33a3b None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:47.839625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:27:47.844516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57696f-5efc-4374-8861-e991269a62a8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:47.844969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57696f-5efc-4374-8861-e991269a62a8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:47.845088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57696f-5efc-4374-8861-e991269a62a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:47.930556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-af57696f-5efc-4374-8861-e991269a62a8 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:27:47.931773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-af57696f-5efc-4374-8861-e991269a62a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:27:48.035490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:27:48.193401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90063cb3-5356-4d22-b070-c06775a33a3b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:27:48.195532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-90063cb3-5356-4d22-b070-c06775a33a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:27:48.315126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90063cb3-5356-4d22-b070-c06775a33a3b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:27:48.320587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8058b-7afa-42b0-9d85-9581f1cf9eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:27:48.321187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8058b-7afa-42b0-9d85-9581f1cf9eb4 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:48.321460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8058b-7afa-42b0-9d85-9581f1cf9eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:48.321728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8058b-7afa-42b0-9d85-9581f1cf9eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:48.329618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8058b-7afa-42b0-9d85-9581f1cf9eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:27:48.331883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8058b-7afa-42b0-9d85-9581f1cf9eb4 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:27:48.341553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 754/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:27:49.178674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60f486-d2b7-4c70-9538-fde3b2f962a5 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:49.179233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60f486-d2b7-4c70-9538-fde3b2f962a5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:49.179233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60f486-d2b7-4c70-9538-fde3b2f962a5 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:49.180599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:27:49.185300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcfa40-b0b5-4560-9079-bbef009c7cd3 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:49.185703 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcfa40-b0b5-4560-9079-bbef009c7cd3 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:49.185855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcfa40-b0b5-4560-9079-bbef009c7cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:49.207084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0abcfa40-b0b5-4560-9079-bbef009c7cd3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:27:49.208274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0abcfa40-b0b5-4560-9079-bbef009c7cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:27:49.216328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:27:49.369685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60f486-d2b7-4c70-9538-fde3b2f962a5 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:27:49.384110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60f486-d2b7-4c70-9538-fde3b2f962a5 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:27:49.389532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a47200-c15a-4ba6-8dae-bb4215d797f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:27:49.390120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a47200-c15a-4ba6-8dae-bb4215d797f7 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:49.390400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a47200-c15a-4ba6-8dae-bb4215d797f7 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:49.390676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a47200-c15a-4ba6-8dae-bb4215d797f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:49.399547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a47200-c15a-4ba6-8dae-bb4215d797f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:27:49.401251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a47200-c15a-4ba6-8dae-bb4215d797f7 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:27:49.411224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 761/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:27:50.359898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a78f01-430d-437e-a0df-eecb765023e0 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:50.360359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a78f01-430d-437e-a0df-eecb765023e0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:50.360463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a78f01-430d-437e-a0df-eecb765023e0 None None] PATH_INFO: `/v3` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:50.362387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 756/1517] 10.222.0.30 () {54 vars in 925 bytes} [Thu Sep 1 00:27:50 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:27:50.383824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c584a92-00c0-4ee3-bafc-b83eb28f32ab None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:50.384273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c584a92-00c0-4ee3-bafc-b83eb28f32ab None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:50.384380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c584a92-00c0-4ee3-bafc-b83eb28f32ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:50.444462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0c584a92-00c0-4ee3-bafc-b83eb28f32ab None None] MFA Rules not processed for user `e30caaea6d104a53a8654442c2383b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:27:50.445724 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0c584a92-00c0-4ee3-bafc-b83eb28f32ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:27:50.462483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:27:50.516613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 762/1518] 10.222.0.30 () {58 vars in 1015 bytes} [Thu Sep 1 00:27:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:27:54.368598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99258c8-7e34-4ef8-9cee-828b3680005e None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:54.369140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99258c8-7e34-4ef8-9cee-828b3680005e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:54.369140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99258c8-7e34-4ef8-9cee-828b3680005e None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:54.370455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 757/1519] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 1 00:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:27:54.375639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec628e-1287-48af-ba28-3d2dd7d3d52b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:54.376061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec628e-1287-48af-ba28-3d2dd7d3d52b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:54.376164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec628e-1287-48af-ba28-3d2dd7d3d52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:54.397443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-fcec628e-1287-48af-ba28-3d2dd7d3d52b None None] MFA Rules not processed for user `e30caaea6d104a53a8654442c2383b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:27:54.398918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fcec628e-1287-48af-ba28-3d2dd7d3d52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:27:54.407972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 763/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 1 00:27:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:27:54.415498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99258c8-7e34-4ef8-9cee-828b3680005e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:27:54.429226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99258c8-7e34-4ef8-9cee-828b3680005e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:27:54.434146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f42336-33e7-4ee3-8f9a-18d96d100862 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:27:54.434737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f42336-33e7-4ee3-8f9a-18d96d100862 service placement] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:27:54.435011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f42336-33e7-4ee3-8f9a-18d96d100862 service placement] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:27:54.435279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f42336-33e7-4ee3-8f9a-18d96d100862 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:27:54.444283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f42336-33e7-4ee3-8f9a-18d96d100862 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:27:54.445895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f42336-33e7-4ee3-8f9a-18d96d100862 service placement] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:27:54.455463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 758/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:27:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:28:00.626841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279f8ee-150a-42cd-af85-cbab9018c816 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:00.627981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279f8ee-150a-42cd-af85-cbab9018c816 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:00.627981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279f8ee-150a-42cd-af85-cbab9018c816 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:00.629353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:28:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:28:00.634088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09528dc7-5c1c-4b92-be3b-8c4a13bfcc72 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:00.634497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09528dc7-5c1c-4b92-be3b-8c4a13bfcc72 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:00.634730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09528dc7-5c1c-4b92-be3b-8c4a13bfcc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:00.653698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-09528dc7-5c1c-4b92-be3b-8c4a13bfcc72 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:28:00.654877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-09528dc7-5c1c-4b92-be3b-8c4a13bfcc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:28:00.662774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:28:00.816396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279f8ee-150a-42cd-af85-cbab9018c816 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:28:00.831009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279f8ee-150a-42cd-af85-cbab9018c816 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:28:00.836602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f26b77-98da-4ffe-9ef0-f611c0a2c420 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:28:00.837195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f26b77-98da-4ffe-9ef0-f611c0a2c420 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:00.837467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f26b77-98da-4ffe-9ef0-f611c0a2c420 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:00.837734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f26b77-98da-4ffe-9ef0-f611c0a2c420 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:00.846607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f26b77-98da-4ffe-9ef0-f611c0a2c420 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:28:00.848207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f26b77-98da-4ffe-9ef0-f611c0a2c420 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:28:00.857490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 765/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:28:01.679656 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436ef46-c04a-4699-bf5e-2c54fd69a1b2 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:01.680220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436ef46-c04a-4699-bf5e-2c54fd69a1b2 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:01.680220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436ef46-c04a-4699-bf5e-2c54fd69a1b2 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:01.681604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 00:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:28:01.686451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c89100-e982-454d-bbb7-40c562a09c32 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:01.686885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c89100-e982-454d-bbb7-40c562a09c32 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:01.686988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c89100-e982-454d-bbb7-40c562a09c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:01.707197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-20c89100-e982-454d-bbb7-40c562a09c32 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:28:01.708479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-20c89100-e982-454d-bbb7-40c562a09c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:28:01.716518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 00:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:28:01.870837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436ef46-c04a-4699-bf5e-2c54fd69a1b2 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:28:01.886343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436ef46-c04a-4699-bf5e-2c54fd69a1b2 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:28:01.891756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5652adc-30f8-4a88-a008-d6e97f15c251 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:28:01.892345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5652adc-30f8-4a88-a008-d6e97f15c251 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:01.892657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5652adc-30f8-4a88-a008-d6e97f15c251 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:01.892957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5652adc-30f8-4a88-a008-d6e97f15c251 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:01.901520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5652adc-30f8-4a88-a008-d6e97f15c251 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:28:01.903118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5652adc-30f8-4a88-a008-d6e97f15c251 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:28:01.912961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 761/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:28:05.581179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40fc1-20ac-4cce-89bd-811bc95abcbe None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:05.581745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40fc1-20ac-4cce-89bd-811bc95abcbe None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:05.581745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40fc1-20ac-4cce-89bd-811bc95abcbe None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:05.583084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:28:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:28:05.588596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1146c0-bed8-47be-bbf3-40dc1e0f486b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:05.589028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1146c0-bed8-47be-bbf3-40dc1e0f486b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:05.589173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1146c0-bed8-47be-bbf3-40dc1e0f486b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:05.608229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-db1146c0-bed8-47be-bbf3-40dc1e0f486b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:28:05.609456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-db1146c0-bed8-47be-bbf3-40dc1e0f486b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:28:05.617853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:28:05.906746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa40fc1-20ac-4cce-89bd-811bc95abcbe None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:28:05.908957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9aa40fc1-20ac-4cce-89bd-811bc95abcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:28:05.987363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa40fc1-20ac-4cce-89bd-811bc95abcbe None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:28:05.990882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fb81c-87c8-4d96-b443-0290636316c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:28:05.991481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fb81c-87c8-4d96-b443-0290636316c3 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:05.991757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fb81c-87c8-4d96-b443-0290636316c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:05.992026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fb81c-87c8-4d96-b443-0290636316c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:06.000685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fb81c-87c8-4d96-b443-0290636316c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:28:06.002182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fb81c-87c8-4d96-b443-0290636316c3 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:28:06.011890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 768/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:28:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:28:07.992453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54862a-c194-4257-82c6-fcad87cb3686 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:07.993082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54862a-c194-4257-82c6-fcad87cb3686 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:07.993082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54862a-c194-4257-82c6-fcad87cb3686 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:07.994388 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:28:07.999541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41d968-fee9-428d-b521-63f538e704a3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:07.999946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41d968-fee9-428d-b521-63f538e704a3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:08.000070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41d968-fee9-428d-b521-63f538e704a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:08.019604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bc41d968-fee9-428d-b521-63f538e704a3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:28:08.020821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bc41d968-fee9-428d-b521-63f538e704a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:28:08.028482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:28:08.311954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54862a-c194-4257-82c6-fcad87cb3686 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:28:08.323346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54862a-c194-4257-82c6-fcad87cb3686 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:28:08.326839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af99a6e-9126-435f-bbb0-4408c0434df4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:28:08.327424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af99a6e-9126-435f-bbb0-4408c0434df4 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:08.327695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af99a6e-9126-435f-bbb0-4408c0434df4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:08.327956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af99a6e-9126-435f-bbb0-4408c0434df4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:08.337409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af99a6e-9126-435f-bbb0-4408c0434df4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:28:08.338826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af99a6e-9126-435f-bbb0-4408c0434df4 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:28:08.348114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 764/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:28:11.725675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d6296-ad9b-4c8f-9ab3-07d5cf7a3449 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:11.726566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d6296-ad9b-4c8f-9ab3-07d5cf7a3449 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:11.726846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31d6296-ad9b-4c8f-9ab3-07d5cf7a3449 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:11.728231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:28:11.733473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f743278-42eb-4d23-948f-3eaa155ecf2a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:11.733873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f743278-42eb-4d23-948f-3eaa155ecf2a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:11.734032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f743278-42eb-4d23-948f-3eaa155ecf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:11.754299 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2f743278-42eb-4d23-948f-3eaa155ecf2a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:28:11.755490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2f743278-42eb-4d23-948f-3eaa155ecf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:28:11.763214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:28:12.314848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d6296-ad9b-4c8f-9ab3-07d5cf7a3449 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:28:12.316511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b31d6296-ad9b-4c8f-9ab3-07d5cf7a3449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:28:12.383516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:28:12.397421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31d6296-ad9b-4c8f-9ab3-07d5cf7a3449 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:28:12.401479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a2ed-0850-4d2b-a87e-b60057d341b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:28:12.402215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a2ed-0850-4d2b-a87e-b60057d341b3 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:12.402597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a2ed-0850-4d2b-a87e-b60057d341b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:12.402961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a2ed-0850-4d2b-a87e-b60057d341b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:12.412870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a2ed-0850-4d2b-a87e-b60057d341b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:28:12.414307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a2ed-0850-4d2b-a87e-b60057d341b3 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:28:12.424147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 771/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:28:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:28:14.330691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14463562-2ffb-4476-87f3-6240f97430a9 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:14.331221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14463562-2ffb-4476-87f3-6240f97430a9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:14.331339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14463562-2ffb-4476-87f3-6240f97430a9 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:14.332664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 00:28:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:28:14.338160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be6762-95ce-41bf-84f5-41f88dc80852 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:14.338562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be6762-95ce-41bf-84f5-41f88dc80852 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:14.338710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be6762-95ce-41bf-84f5-41f88dc80852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:14.357796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e4be6762-95ce-41bf-84f5-41f88dc80852 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:28:14.358997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e4be6762-95ce-41bf-84f5-41f88dc80852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:28:14.366785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 00:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:28:14.924230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14463562-2ffb-4476-87f3-6240f97430a9 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:28:14.925947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-14463562-2ffb-4476-87f3-6240f97430a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:28:14.933418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=84 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:28:14.940238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14463562-2ffb-4476-87f3-6240f97430a9 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:28:14.943850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bc25d-e26e-450b-a72b-7bdc915e7ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:28:14.944456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bc25d-e26e-450b-a72b-7bdc915e7ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:28:14.944768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bc25d-e26e-450b-a72b-7bdc915e7ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:28:14.945044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bc25d-e26e-450b-a72b-7bdc915e7ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:28:14.954016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274bc25d-e26e-450b-a72b-7bdc915e7ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:28:14.955477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274bc25d-e26e-450b-a72b-7bdc915e7ab5 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:28:14.965018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 767/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:28:24.352581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=11 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:28:24.960904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:40.249176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b688778-5a2b-4ee8-8e8f-fa0b07dd8d5c None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.250247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b688778-5a2b-4ee8-8e8f-fa0b07dd8d5c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.250748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b688778-5a2b-4ee8-8e8f-fa0b07dd8d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.276164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a4dac-7321-4aac-962b-f0b13e31bb2c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.277229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a4dac-7321-4aac-962b-f0b13e31bb2c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.277721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a4dac-7321-4aac-962b-f0b13e31bb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.280230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-9b688778-5a2b-4ee8-8e8f-fa0b07dd8d5c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.284520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9b688778-5a2b-4ee8-8e8f-fa0b07dd8d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.296538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 773/1540] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.300368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f89e71-fcbe-4e18-86e4-829449e93de7 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.300882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f89e71-fcbe-4e18-86e4-829449e93de7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.301325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f89e71-fcbe-4e18-86e4-829449e93de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.317202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-672a4dac-7321-4aac-962b-f0b13e31bb2c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.318431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-672a4dac-7321-4aac-962b-f0b13e31bb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.334286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f4f89e71-fcbe-4e18-86e4-829449e93de7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.335889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f4f89e71-fcbe-4e18-86e4-829449e93de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.344526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 768/1541] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.351909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 774/1542] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:29:40.356773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f89e71-fcbe-4e18-86e4-829449e93de7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.357511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb998470-2616-4aa7-a392-1835fe65b7e4 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.357805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb998470-2616-4aa7-a392-1835fe65b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.358081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb998470-2616-4aa7-a392-1835fe65b7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.372521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f89e71-fcbe-4e18-86e4-829449e93de7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.378047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dbc3b-deae-4eb7-ae1f-8ba1155e7603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.378930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dbc3b-deae-4eb7-ae1f-8ba1155e7603 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.379395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dbc3b-deae-4eb7-ae1f-8ba1155e7603 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.379859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dbc3b-deae-4eb7-ae1f-8ba1155e7603 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.385993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dbc3b-deae-4eb7-ae1f-8ba1155e7603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:40.386586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dbc3b-deae-4eb7-ae1f-8ba1155e7603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.388441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dbc3b-deae-4eb7-ae1f-8ba1155e7603 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.389552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fb998470-2616-4aa7-a392-1835fe65b7e4 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.395461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fb998470-2616-4aa7-a392-1835fe65b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.416355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 775/1543] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:40.420437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 769/1544] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.421650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dbc3b-deae-4eb7-ae1f-8ba1155e7603 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.427674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb998470-2616-4aa7-a392-1835fe65b7e4 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.444177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb998470-2616-4aa7-a392-1835fe65b7e4 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.448143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dbc3b-deae-4eb7-ae1f-8ba1155e7603 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.454918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8f22c-45ca-4785-b0a5-196d9acb674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.455552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f22c-45ca-4785-b0a5-196d9acb674d admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.455828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f22c-45ca-4785-b0a5-196d9acb674d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.456219 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8f22c-45ca-4785-b0a5-196d9acb674d admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.457414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8f22c-45ca-4785-b0a5-196d9acb674d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.459177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8f22c-45ca-4785-b0a5-196d9acb674d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.459786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665965be-020e-4939-bb01-0f0a68da4167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.460692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665965be-020e-4939-bb01-0f0a68da4167 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.461190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665965be-020e-4939-bb01-0f0a68da4167 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.461678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665965be-020e-4939-bb01-0f0a68da4167 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.462526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665965be-020e-4939-bb01-0f0a68da4167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:40.463047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665965be-020e-4939-bb01-0f0a68da4167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.464886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665965be-020e-4939-bb01-0f0a68da4167 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.490440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 776/1545] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:40.491734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45d421-d4ad-424c-9b42-aa56d1547a1f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.492015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45d421-d4ad-424c-9b42-aa56d1547a1f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.492284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45d421-d4ad-424c-9b42-aa56d1547a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.514870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ec45d421-d4ad-424c-9b42-aa56d1547a1f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.516071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ec45d421-d4ad-424c-9b42-aa56d1547a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.521331 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 770/1546] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/projects => generated 391 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:40.522937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9dce-8e35-4dc6-adaa-8fa73af718e6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.523274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9dce-8e35-4dc6-adaa-8fa73af718e6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.523553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9dce-8e35-4dc6-adaa-8fa73af718e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.528619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 777/1547] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.537946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093808e-0089-41a2-a9a3-dbd2f5b5dc51 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.538304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093808e-0089-41a2-a9a3-dbd2f5b5dc51 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.538516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093808e-0089-41a2-a9a3-dbd2f5b5dc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.550426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2c9a9dce-8e35-4dc6-adaa-8fa73af718e6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.551810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2c9a9dce-8e35-4dc6-adaa-8fa73af718e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.562312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-2093808e-0089-41a2-a9a3-dbd2f5b5dc51 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.563527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2093808e-0089-41a2-a9a3-dbd2f5b5dc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.622312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 771/1548] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.623644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a0bcf-484b-42f2-8f56-9206c29f6af6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.623953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a0bcf-484b-42f2-8f56-9206c29f6af6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.624223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a0bcf-484b-42f2-8f56-9206c29f6af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.636827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 778/1549] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.637661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093808e-0089-41a2-a9a3-dbd2f5b5dc51 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.643855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b97a0bcf-484b-42f2-8f56-9206c29f6af6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.645164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b97a0bcf-484b-42f2-8f56-9206c29f6af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.647401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093808e-0089-41a2-a9a3-dbd2f5b5dc51 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.651882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c4557-d7f0-4088-a0ae-5900dc1aef15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.652475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c4557-d7f0-4088-a0ae-5900dc1aef15 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.652784 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c4557-d7f0-4088-a0ae-5900dc1aef15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.653056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c4557-d7f0-4088-a0ae-5900dc1aef15 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.653209 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 772/1550] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.653685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c4557-d7f0-4088-a0ae-5900dc1aef15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.653970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a0bcf-484b-42f2-8f56-9206c29f6af6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.654707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING py.warnings [None req-3c4c4557-d7f0-4088-a0ae-5900dc1aef15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:29:40.654707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: warnings.warn(msg) Sep 01 00:29:40.654707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: Sep 01 00:29:40.655893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c4557-d7f0-4088-a0ae-5900dc1aef15 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.664801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a0bcf-484b-42f2-8f56-9206c29f6af6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.669415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43671037-00a4-435b-88d5-de07a17b6e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.670038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671037-00a4-435b-88d5-de07a17b6e22 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.670313 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671037-00a4-435b-88d5-de07a17b6e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.670579 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671037-00a4-435b-88d5-de07a17b6e22 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.671195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43671037-00a4-435b-88d5-de07a17b6e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.672217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WARNING py.warnings [None req-43671037-00a4-435b-88d5-de07a17b6e22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:29:40.672217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: warnings.warn(msg) Sep 01 00:29:40.672217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: Sep 01 00:29:40.674055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43671037-00a4-435b-88d5-de07a17b6e22 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.686393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 779/1551] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:40.687653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff122ba-5512-480e-a572-e2316e6b19d8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.687933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff122ba-5512-480e-a572-e2316e6b19d8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.688207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff122ba-5512-480e-a572-e2316e6b19d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.699210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 773/1552] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/users => generated 296 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:40.700837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba99bbd-f0ee-4feb-b71a-31ec54d9115e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.701308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba99bbd-f0ee-4feb-b71a-31ec54d9115e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.701582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba99bbd-f0ee-4feb-b71a-31ec54d9115e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.709422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-4ff122ba-5512-480e-a572-e2316e6b19d8 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.710628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4ff122ba-5512-480e-a572-e2316e6b19d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.722721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-eba99bbd-f0ee-4feb-b71a-31ec54d9115e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.723940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-eba99bbd-f0ee-4feb-b71a-31ec54d9115e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.772878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 780/1553] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.774069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b7702-0072-4825-ae3c-0d325e6db8a1 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.774355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b7702-0072-4825-ae3c-0d325e6db8a1 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.774629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b7702-0072-4825-ae3c-0d325e6db8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.787053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 774/1554] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.788300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26190757-50d2-40e1-afdd-3295268e3e4a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.788628 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26190757-50d2-40e1-afdd-3295268e3e4a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.788904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26190757-50d2-40e1-afdd-3295268e3e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.795595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d42b7702-0072-4825-ae3c-0d325e6db8a1 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.796837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d42b7702-0072-4825-ae3c-0d325e6db8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.805396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 781/1555] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.806584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cee70-18fc-417c-a455-e0136c1a0ee7 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.806858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cee70-18fc-417c-a455-e0136c1a0ee7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.807126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cee70-18fc-417c-a455-e0136c1a0ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.810379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-26190757-50d2-40e1-afdd-3295268e3e4a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.811595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-26190757-50d2-40e1-afdd-3295268e3e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.820124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 775/1556] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.821387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f876e0d-4506-440c-be77-49ac5334aab7 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.821667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f876e0d-4506-440c-be77-49ac5334aab7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.821938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f876e0d-4506-440c-be77-49ac5334aab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.830673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-223cee70-18fc-417c-a455-e0136c1a0ee7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.832002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-223cee70-18fc-417c-a455-e0136c1a0ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.840209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 782/1557] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.841110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cee70-18fc-417c-a455-e0136c1a0ee7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.842357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3f876e0d-4506-440c-be77-49ac5334aab7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.843603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3f876e0d-4506-440c-be77-49ac5334aab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.852108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 776/1558] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.852634 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cee70-18fc-417c-a455-e0136c1a0ee7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.853052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f876e0d-4506-440c-be77-49ac5334aab7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.857789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80831624-fbe3-4c30-89b0-3c712f966ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.858381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80831624-fbe3-4c30-89b0-3c712f966ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.858660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80831624-fbe3-4c30-89b0-3c712f966ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.858929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80831624-fbe3-4c30-89b0-3c712f966ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.859553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80831624-fbe3-4c30-89b0-3c712f966ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.862133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80831624-fbe3-4c30-89b0-3c712f966ec9 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.863251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f876e0d-4506-440c-be77-49ac5334aab7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.867866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c70dcd-fdef-4fcc-b9bf-0266608bf6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.868466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c70dcd-fdef-4fcc-b9bf-0266608bf6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.868780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c70dcd-fdef-4fcc-b9bf-0266608bf6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.869049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c70dcd-fdef-4fcc-b9bf-0266608bf6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.869690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c70dcd-fdef-4fcc-b9bf-0266608bf6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.871284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c70dcd-fdef-4fcc-b9bf-0266608bf6c0 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.882048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 777/1559] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:40.882861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c70dcd-fdef-4fcc-b9bf-0266608bf6c0 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.886557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 783/1560] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:40.886557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4272b7-c302-4236-83a4-db82f0138b17 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.886557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4272b7-c302-4236-83a4-db82f0138b17 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.886557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4272b7-c302-4236-83a4-db82f0138b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.893361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c70dcd-fdef-4fcc-b9bf-0266608bf6c0 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.894384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:40.899410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b24c5c-0fdf-479e-884a-d9d796bd34c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.899874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b24c5c-0fdf-479e-884a-d9d796bd34c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.900147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b24c5c-0fdf-479e-884a-d9d796bd34c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.900414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b24c5c-0fdf-479e-884a-d9d796bd34c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.901079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b24c5c-0fdf-479e-884a-d9d796bd34c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:40.901376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b24c5c-0fdf-479e-884a-d9d796bd34c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.902376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WARNING py.warnings [None req-b7b24c5c-0fdf-479e-884a-d9d796bd34c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:29:40.902376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: warnings.warn(msg) Sep 01 00:29:40.902376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: Sep 01 00:29:40.903251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b24c5c-0fdf-479e-884a-d9d796bd34c5 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.905939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:40.909272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-fd4272b7-c302-4236-83a4-db82f0138b17 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.910487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fd4272b7-c302-4236-83a4-db82f0138b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.917003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 778/1561] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:40.918864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e9578-738c-4fd0-b132-d9da12f0f1e7 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.919151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e9578-738c-4fd0-b132-d9da12f0f1e7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.919430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e9578-738c-4fd0-b132-d9da12f0f1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.922985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 784/1562] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.925347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4272b7-c302-4236-83a4-db82f0138b17 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.934873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4272b7-c302-4236-83a4-db82f0138b17 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.939747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e1b3a-1447-4b25-a618-d64026cf2263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.940192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e47e9578-738c-4fd0-b132-d9da12f0f1e7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:40.940640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1b3a-1447-4b25-a618-d64026cf2263 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.940981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1b3a-1447-4b25-a618-d64026cf2263 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.941312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e1b3a-1447-4b25-a618-d64026cf2263 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.941459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e47e9578-738c-4fd0-b132-d9da12f0f1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:40.942063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e1b3a-1447-4b25-a618-d64026cf2263 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:40.942435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e1b3a-1447-4b25-a618-d64026cf2263 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.944094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e1b3a-1447-4b25-a618-d64026cf2263 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.949410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 779/1563] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:40.950196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e9578-738c-4fd0-b132-d9da12f0f1e7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.956948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 785/1564] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:40.957844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e1b3a-1447-4b25-a618-d64026cf2263 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.961016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e9578-738c-4fd0-b132-d9da12f0f1e7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.965619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03622b-fcee-49dc-99c1-a03bfbc3a9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.966290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03622b-fcee-49dc-99c1-a03bfbc3a9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.966494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03622b-fcee-49dc-99c1-a03bfbc3a9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.966766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03622b-fcee-49dc-99c1-a03bfbc3a9a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.967439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03622b-fcee-49dc-99c1-a03bfbc3a9a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:40.967744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03622b-fcee-49dc-99c1-a03bfbc3a9a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.968215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e1b3a-1447-4b25-a618-d64026cf2263 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:40.969404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03622b-fcee-49dc-99c1-a03bfbc3a9a6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.973013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d053-60a9-4add-afd1-d886c9ea956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:40.973587 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d053-60a9-4add-afd1-d886c9ea956f admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:40.973857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d053-60a9-4add-afd1-d886c9ea956f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:40.974122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d053-60a9-4add-afd1-d886c9ea956f admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:40.974749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d053-60a9-4add-afd1-d886c9ea956f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:40.975042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d053-60a9-4add-afd1-d886c9ea956f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:40.976647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d053-60a9-4add-afd1-d886c9ea956f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:40.981237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 780/1565] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:40.982043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03622b-fcee-49dc-99c1-a03bfbc3a9a6 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.989830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 786/1566] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:40.990729 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d053-60a9-4add-afd1-d886c9ea956f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:40.995907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03622b-fcee-49dc-99c1-a03bfbc3a9a6 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.000235 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d053-60a9-4add-afd1-d886c9ea956f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.000819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdb817-432d-48a7-88e4-847b8f36170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.001421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdb817-432d-48a7-88e4-847b8f36170d admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.001759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdb817-432d-48a7-88e4-847b8f36170d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.002039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdb817-432d-48a7-88e4-847b8f36170d admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.002684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdb817-432d-48a7-88e4-847b8f36170d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.004319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdb817-432d-48a7-88e4-847b8f36170d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.005324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a5093-57e3-461b-9dcc-94ffa215f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.005901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a5093-57e3-461b-9dcc-94ffa215f7fb admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.006178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a5093-57e3-461b-9dcc-94ffa215f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.006453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a5093-57e3-461b-9dcc-94ffa215f7fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.007081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a5093-57e3-461b-9dcc-94ffa215f7fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.008689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a5093-57e3-461b-9dcc-94ffa215f7fb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.015497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 781/1567] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:41.016269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdb817-432d-48a7-88e4-847b8f36170d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.022343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 787/1568] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:41.023213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a5093-57e3-461b-9dcc-94ffa215f7fb admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.026321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdb817-432d-48a7-88e4-847b8f36170d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.030886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ea7c4-072c-4c5d-b98d-c54c7645c3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.031467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ea7c4-072c-4c5d-b98d-c54c7645c3bb admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.031743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ea7c4-072c-4c5d-b98d-c54c7645c3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.032013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ea7c4-072c-4c5d-b98d-c54c7645c3bb admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.032507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a5093-57e3-461b-9dcc-94ffa215f7fb admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.032836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ea7c4-072c-4c5d-b98d-c54c7645c3bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.034885 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ea7c4-072c-4c5d-b98d-c54c7645c3bb admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.037430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf96f78-05dd-4ec7-93e9-fd5d5a5dab7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.038008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf96f78-05dd-4ec7-93e9-fd5d5a5dab7d admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.038286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf96f78-05dd-4ec7-93e9-fd5d5a5dab7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.038692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf96f78-05dd-4ec7-93e9-fd5d5a5dab7d admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.039369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf96f78-05dd-4ec7-93e9-fd5d5a5dab7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:41.039673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf96f78-05dd-4ec7-93e9-fd5d5a5dab7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.041338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf96f78-05dd-4ec7-93e9-fd5d5a5dab7d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.054068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 788/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:41.055015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf96f78-05dd-4ec7-93e9-fd5d5a5dab7d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.071040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf96f78-05dd-4ec7-93e9-fd5d5a5dab7d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.075157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 782/1570] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.075157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ea7c4-072c-4c5d-b98d-c54c7645c3bb admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.131951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6aaaa4-fe53-47ad-9ec2-99404d52ae88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.132589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6aaaa4-fe53-47ad-9ec2-99404d52ae88 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.132871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6aaaa4-fe53-47ad-9ec2-99404d52ae88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.133140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6aaaa4-fe53-47ad-9ec2-99404d52ae88 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.133780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6aaaa4-fe53-47ad-9ec2-99404d52ae88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:41.134079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6aaaa4-fe53-47ad-9ec2-99404d52ae88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.135717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6aaaa4-fe53-47ad-9ec2-99404d52ae88 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.142698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ea7c4-072c-4c5d-b98d-c54c7645c3bb admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.147374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0c860-7126-456f-8f73-e07f203797c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.147538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 789/1571] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:41.148272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0c860-7126-456f-8f73-e07f203797c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.148397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6aaaa4-fe53-47ad-9ec2-99404d52ae88 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.148593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0c860-7126-456f-8f73-e07f203797c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.148848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0c860-7126-456f-8f73-e07f203797c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.149470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0c860-7126-456f-8f73-e07f203797c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.151379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0c860-7126-456f-8f73-e07f203797c3 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.165241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6aaaa4-fe53-47ad-9ec2-99404d52ae88 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.170041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced13eb0-5dbe-456b-9779-34a2bccfa11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.170633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced13eb0-5dbe-456b-9779-34a2bccfa11e admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.171023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced13eb0-5dbe-456b-9779-34a2bccfa11e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.171478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced13eb0-5dbe-456b-9779-34a2bccfa11e admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.172104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced13eb0-5dbe-456b-9779-34a2bccfa11e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.174879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced13eb0-5dbe-456b-9779-34a2bccfa11e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.182771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 783/1572] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.182771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0c860-7126-456f-8f73-e07f203797c3 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.205963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 790/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.206765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced13eb0-5dbe-456b-9779-34a2bccfa11e admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.249426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0c860-7126-456f-8f73-e07f203797c3 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.271719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced13eb0-5dbe-456b-9779-34a2bccfa11e admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.276486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f4c63-cb6c-482b-ae4c-1197db9a4fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.277109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f4c63-cb6c-482b-ae4c-1197db9a4fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.277389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f4c63-cb6c-482b-ae4c-1197db9a4fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.277660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f4c63-cb6c-482b-ae4c-1197db9a4fb8 admin admin] PATH_INFO: `/v3/projects/64e86923fd7b42cfaf18408210d4f6d7/users/a6f08eaef2d149a9b0236d4bcbe03a92/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.284778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:41.293236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f4c63-cb6c-482b-ae4c-1197db9a4fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e86923fd7b42cfaf18408210d4f6d7, user_id=a6f08eaef2d149a9b0236d4bcbe03a92, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.294284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING py.warnings [None req-b62f4c63-cb6c-482b-ae4c-1197db9a4fb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:29:41.294284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: warnings.warn(msg) Sep 01 00:29:41.294284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: Sep 01 00:29:41.296415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f4c63-cb6c-482b-ae4c-1197db9a4fb8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.306142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:41.311382 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 791/1574] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:41 2022] PUT /identity/v3/projects/64e86923fd7b42cfaf18408210d4f6d7/users/a6f08eaef2d149a9b0236d4bcbe03a92/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:41.312467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a51f0c-0445-4325-9a0d-2405c13d74d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.312645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f4c63-cb6c-482b-ae4c-1197db9a4fb8 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.313260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a51f0c-0445-4325-9a0d-2405c13d74d8 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.313545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a51f0c-0445-4325-9a0d-2405c13d74d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.313820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a51f0c-0445-4325-9a0d-2405c13d74d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.315949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a51f0c-0445-4325-9a0d-2405c13d74d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.317930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a51f0c-0445-4325-9a0d-2405c13d74d8 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.346740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 784/1575] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.348301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a51f0c-0445-4325-9a0d-2405c13d74d8 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.385096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f4c63-cb6c-482b-ae4c-1197db9a4fb8 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.413948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a51f0c-0445-4325-9a0d-2405c13d74d8 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.418575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac3dcd-6c2a-425b-a6e8-2ed892977dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.420781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac3dcd-6c2a-425b-a6e8-2ed892977dcd admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.421975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac3dcd-6c2a-425b-a6e8-2ed892977dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.422406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac3dcd-6c2a-425b-a6e8-2ed892977dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.423208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac3dcd-6c2a-425b-a6e8-2ed892977dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.427381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac3dcd-6c2a-425b-a6e8-2ed892977dcd admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.447643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d5a1a-1069-4b4a-a203-80cc5911b0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.448432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d5a1a-1069-4b4a-a203-80cc5911b0db admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.448767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d5a1a-1069-4b4a-a203-80cc5911b0db admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.449051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d5a1a-1069-4b4a-a203-80cc5911b0db admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.449685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d5a1a-1069-4b4a-a203-80cc5911b0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.451316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d5a1a-1069-4b4a-a203-80cc5911b0db admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.456562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 785/1576] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.457868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac3dcd-6c2a-425b-a6e8-2ed892977dcd admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.465568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 792/1577] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:41.465568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d5a1a-1069-4b4a-a203-80cc5911b0db admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.472988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d5a1a-1069-4b4a-a203-80cc5911b0db admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.522305 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac3dcd-6c2a-425b-a6e8-2ed892977dcd admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.526985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef167613-0f0e-4cf5-976a-edd3f40ba001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.527527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef167613-0f0e-4cf5-976a-edd3f40ba001 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.527802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef167613-0f0e-4cf5-976a-edd3f40ba001 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.528131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef167613-0f0e-4cf5-976a-edd3f40ba001 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.528973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef167613-0f0e-4cf5-976a-edd3f40ba001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.530999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef167613-0f0e-4cf5-976a-edd3f40ba001 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.533338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37307e30-0c64-451c-8250-930238cec03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.533947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37307e30-0c64-451c-8250-930238cec03d admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.534227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37307e30-0c64-451c-8250-930238cec03d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.534501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37307e30-0c64-451c-8250-930238cec03d admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.535127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37307e30-0c64-451c-8250-930238cec03d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.537061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37307e30-0c64-451c-8250-930238cec03d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.552743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 786/1578] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.553536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef167613-0f0e-4cf5-976a-edd3f40ba001 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.568954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 793/1579] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.569151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef167613-0f0e-4cf5-976a-edd3f40ba001 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.569241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37307e30-0c64-451c-8250-930238cec03d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.596490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:41.598372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:41.625678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284f49d-9cff-4071-a2a2-56cf0aba5c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.626296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284f49d-9cff-4071-a2a2-56cf0aba5c63 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.627116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284f49d-9cff-4071-a2a2-56cf0aba5c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.627401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284f49d-9cff-4071-a2a2-56cf0aba5c63 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.628127 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284f49d-9cff-4071-a2a2-56cf0aba5c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.630078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284f49d-9cff-4071-a2a2-56cf0aba5c63 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.631083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37307e30-0c64-451c-8250-930238cec03d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.635644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61722298-85fa-4f30-8a9c-c485d6cc5e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.636263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61722298-85fa-4f30-8a9c-c485d6cc5e36 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.636571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61722298-85fa-4f30-8a9c-c485d6cc5e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.636843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61722298-85fa-4f30-8a9c-c485d6cc5e36 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.637456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61722298-85fa-4f30-8a9c-c485d6cc5e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.640460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61722298-85fa-4f30-8a9c-c485d6cc5e36 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.651613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 787/1580] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.652959 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989a1c4-6144-40c5-982c-d5b410510839 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.653242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989a1c4-6144-40c5-982c-d5b410510839 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.653513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989a1c4-6144-40c5-982c-d5b410510839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.668747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 794/1581] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.668747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61722298-85fa-4f30-8a9c-c485d6cc5e36 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.674173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2989a1c4-6144-40c5-982c-d5b410510839 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:41.675668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2989a1c4-6144-40c5-982c-d5b410510839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:41.676278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61722298-85fa-4f30-8a9c-c485d6cc5e36 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.680797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d402ca-e5da-451a-965b-006d9b75b290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.681389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d402ca-e5da-451a-965b-006d9b75b290 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.681665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d402ca-e5da-451a-965b-006d9b75b290 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.681991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d402ca-e5da-451a-965b-006d9b75b290 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.682580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d402ca-e5da-451a-965b-006d9b75b290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.686349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 788/1582] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:41.686349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989a1c4-6144-40c5-982c-d5b410510839 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.686793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d402ca-e5da-451a-965b-006d9b75b290 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.694961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989a1c4-6144-40c5-982c-d5b410510839 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.701246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e348c-69d7-4296-a030-3b21446b24d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.701907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e348c-69d7-4296-a030-3b21446b24d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.702184 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e348c-69d7-4296-a030-3b21446b24d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.702452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e348c-69d7-4296-a030-3b21446b24d4 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.703089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e348c-69d7-4296-a030-3b21446b24d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.705327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e348c-69d7-4296-a030-3b21446b24d4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.710020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 795/1583] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.710705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d402ca-e5da-451a-965b-006d9b75b290 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.723053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d402ca-e5da-451a-965b-006d9b75b290 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.728741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183d572-53da-4634-ab9d-e4afb208ee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.729401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183d572-53da-4634-ab9d-e4afb208ee77 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.729790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183d572-53da-4634-ab9d-e4afb208ee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.730094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183d572-53da-4634-ab9d-e4afb208ee77 admin admin] PATH_INFO: `/v3/projects/cb964432f4ad4e0c8d38d9449ebfaf41/users/67dd50d6270248f298233ee15d2cd23a/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.733833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 789/1584] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.733833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e348c-69d7-4296-a030-3b21446b24d4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.742624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e348c-69d7-4296-a030-3b21446b24d4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.745777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183d572-53da-4634-ab9d-e4afb208ee77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb964432f4ad4e0c8d38d9449ebfaf41, user_id=67dd50d6270248f298233ee15d2cd23a, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.747214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7645c5-3ac3-41fa-aaf7-23625cac9574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.747801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7645c5-3ac3-41fa-aaf7-23625cac9574 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.748076 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7645c5-3ac3-41fa-aaf7-23625cac9574 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.748344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7645c5-3ac3-41fa-aaf7-23625cac9574 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.748597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183d572-53da-4634-ab9d-e4afb208ee77 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.749108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7645c5-3ac3-41fa-aaf7-23625cac9574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.750682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7645c5-3ac3-41fa-aaf7-23625cac9574 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.762124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 790/1585] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:41.763032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 796/1586] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:41 2022] PUT /identity/v3/projects/cb964432f4ad4e0c8d38d9449ebfaf41/users/67dd50d6270248f298233ee15d2cd23a/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:41.765500 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7645c5-3ac3-41fa-aaf7-23625cac9574 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.769487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183d572-53da-4634-ab9d-e4afb208ee77 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.776273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7645c5-3ac3-41fa-aaf7-23625cac9574 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.832119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183d572-53da-4634-ab9d-e4afb208ee77 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.834622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a77ed-fc60-4426-acf6-d02b103da846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.835232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a77ed-fc60-4426-acf6-d02b103da846 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.835549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a77ed-fc60-4426-acf6-d02b103da846 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.835773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a77ed-fc60-4426-acf6-d02b103da846 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.836394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a77ed-fc60-4426-acf6-d02b103da846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.837177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8867232-a33d-4356-86dc-4fd57d9abd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.837797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8867232-a33d-4356-86dc-4fd57d9abd92 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.838076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8867232-a33d-4356-86dc-4fd57d9abd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.838330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8867232-a33d-4356-86dc-4fd57d9abd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.838760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a77ed-fc60-4426-acf6-d02b103da846 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.838970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8867232-a33d-4356-86dc-4fd57d9abd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.840604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8867232-a33d-4356-86dc-4fd57d9abd92 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.850886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 797/1587] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:41.851702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8867232-a33d-4356-86dc-4fd57d9abd92 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.861943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8867232-a33d-4356-86dc-4fd57d9abd92 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.866789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 791/1588] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:41.866789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a77ed-fc60-4426-acf6-d02b103da846 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.867036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62139e18-f2fa-4033-afd4-6ef40bf0ede7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.867183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62139e18-f2fa-4033-afd4-6ef40bf0ede7 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.867489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62139e18-f2fa-4033-afd4-6ef40bf0ede7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.867759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62139e18-f2fa-4033-afd4-6ef40bf0ede7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.868388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62139e18-f2fa-4033-afd4-6ef40bf0ede7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.870017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62139e18-f2fa-4033-afd4-6ef40bf0ede7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.883438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 798/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:41.883438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62139e18-f2fa-4033-afd4-6ef40bf0ede7 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.892686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62139e18-f2fa-4033-afd4-6ef40bf0ede7 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.897623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fda38-7ce6-4515-abef-1518fd7111de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.898193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fda38-7ce6-4515-abef-1518fd7111de admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.898462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fda38-7ce6-4515-abef-1518fd7111de admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.898726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fda38-7ce6-4515-abef-1518fd7111de admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.899340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fda38-7ce6-4515-abef-1518fd7111de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.901001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fda38-7ce6-4515-abef-1518fd7111de admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.912013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 799/1590] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:41.912834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fda38-7ce6-4515-abef-1518fd7111de admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.921424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a77ed-fc60-4426-acf6-d02b103da846 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.922251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fda38-7ce6-4515-abef-1518fd7111de admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.925699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f0f01-7dc8-4ecf-8e9c-9e289a45879f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.926314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f0f01-7dc8-4ecf-8e9c-9e289a45879f service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.926596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f0f01-7dc8-4ecf-8e9c-9e289a45879f service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.926873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f0f01-7dc8-4ecf-8e9c-9e289a45879f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.928024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447a94e-62cb-4135-af40-cd92edbe77d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.928684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447a94e-62cb-4135-af40-cd92edbe77d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.928959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447a94e-62cb-4135-af40-cd92edbe77d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.929226 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447a94e-62cb-4135-af40-cd92edbe77d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.929848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447a94e-62cb-4135-af40-cd92edbe77d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.931453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447a94e-62cb-4135-af40-cd92edbe77d1 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.935986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f0f01-7dc8-4ecf-8e9c-9e289a45879f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.937470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f0f01-7dc8-4ecf-8e9c-9e289a45879f service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.941691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 800/1591] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:41.942521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447a94e-62cb-4135-af40-cd92edbe77d1 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.950131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 792/1592] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:41.951451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7e979-8b52-4df9-a64f-8732c8fe30f9 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.951668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447a94e-62cb-4135-af40-cd92edbe77d1 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.951785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7e979-8b52-4df9-a64f-8732c8fe30f9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.952008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7e979-8b52-4df9-a64f-8732c8fe30f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.956490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67a57a-52c5-437d-8c1c-8a608b9b157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.957101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67a57a-52c5-437d-8c1c-8a608b9b157f admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.957376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67a57a-52c5-437d-8c1c-8a608b9b157f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.957647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67a57a-52c5-437d-8c1c-8a608b9b157f admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.958277 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67a57a-52c5-437d-8c1c-8a608b9b157f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.960249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67a57a-52c5-437d-8c1c-8a608b9b157f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:41.971392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 801/1593] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:41.972854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67a57a-52c5-437d-8c1c-8a608b9b157f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:41.984332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67a57a-52c5-437d-8c1c-8a608b9b157f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:41.990687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60034ce-6c1d-4289-b074-966f25a4226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:41.991285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60034ce-6c1d-4289-b074-966f25a4226b admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:41.991561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60034ce-6c1d-4289-b074-966f25a4226b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:41.991829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60034ce-6c1d-4289-b074-966f25a4226b admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:41.992241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-35e7e979-8b52-4df9-a64f-8732c8fe30f9 None None] MFA Rules not processed for user `67dd50d6270248f298233ee15d2cd23a`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:41.992673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60034ce-6c1d-4289-b074-966f25a4226b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:41.994101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-35e7e979-8b52-4df9-a64f-8732c8fe30f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:41.994447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60034ce-6c1d-4289-b074-966f25a4226b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.010230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 802/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:42.010230 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60034ce-6c1d-4289-b074-966f25a4226b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.012757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:42.022147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60034ce-6c1d-4289-b074-966f25a4226b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.027279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84abfb1-c1de-4587-a0a0-6b7f750ad975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.027892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84abfb1-c1de-4587-a0a0-6b7f750ad975 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.028161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84abfb1-c1de-4587-a0a0-6b7f750ad975 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.028452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84abfb1-c1de-4587-a0a0-6b7f750ad975 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.029144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84abfb1-c1de-4587-a0a0-6b7f750ad975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.030741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84abfb1-c1de-4587-a0a0-6b7f750ad975 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.043403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 803/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:42.047155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84abfb1-c1de-4587-a0a0-6b7f750ad975 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.058188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84abfb1-c1de-4587-a0a0-6b7f750ad975 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.063443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764375ed-1d02-4bb4-83bf-1c1e421486f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.064108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764375ed-1d02-4bb4-83bf-1c1e421486f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.064392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764375ed-1d02-4bb4-83bf-1c1e421486f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.064785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764375ed-1d02-4bb4-83bf-1c1e421486f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.065336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764375ed-1d02-4bb4-83bf-1c1e421486f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.066952 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764375ed-1d02-4bb4-83bf-1c1e421486f7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.068606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:42.084086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 793/1596] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:42.084450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 804/1597] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:42.085569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764375ed-1d02-4bb4-83bf-1c1e421486f7 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.087510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7e979-8b52-4df9-a64f-8732c8fe30f9 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.097199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764375ed-1d02-4bb4-83bf-1c1e421486f7 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.100096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7e979-8b52-4df9-a64f-8732c8fe30f9 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.102018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb749fa-b5c1-4239-9efc-f2de641e96f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.102621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb749fa-b5c1-4239-9efc-f2de641e96f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.102904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb749fa-b5c1-4239-9efc-f2de641e96f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.103181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb749fa-b5c1-4239-9efc-f2de641e96f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.103874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb749fa-b5c1-4239-9efc-f2de641e96f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.105479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb749fa-b5c1-4239-9efc-f2de641e96f4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.105608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25b5f-e090-42e3-8d2a-c85412c9be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.106274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25b5f-e090-42e3-8d2a-c85412c9be34 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.106635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25b5f-e090-42e3-8d2a-c85412c9be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.107116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa25b5f-e090-42e3-8d2a-c85412c9be34 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.108177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa25b5f-e090-42e3-8d2a-c85412c9be34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.109948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa25b5f-e090-42e3-8d2a-c85412c9be34 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.117539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 805/1598] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:42.118231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb749fa-b5c1-4239-9efc-f2de641e96f4 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.128969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb749fa-b5c1-4239-9efc-f2de641e96f4 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.130268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 794/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:42.131341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25b5f-e090-42e3-8d2a-c85412c9be34 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.136872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82c810-a082-4d3b-b97a-7a663b109683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.136872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82c810-a082-4d3b-b97a-7a663b109683 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.137647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82c810-a082-4d3b-b97a-7a663b109683 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.137647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82c810-a082-4d3b-b97a-7a663b109683 admin admin] PATH_INFO: `/v3/projects/becec5cd5afd44848e7178d7a6c2150d/users/94ee943e661d4eb4a860fcea5db842b0/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.159587 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82c810-a082-4d3b-b97a-7a663b109683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=becec5cd5afd44848e7178d7a6c2150d, user_id=94ee943e661d4eb4a860fcea5db842b0, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.161666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa25b5f-e090-42e3-8d2a-c85412c9be34 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.162772 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82c810-a082-4d3b-b97a-7a663b109683 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.167752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2178bc-b91e-4693-8a9f-3497e51e0491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.168517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2178bc-b91e-4693-8a9f-3497e51e0491 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.168820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2178bc-b91e-4693-8a9f-3497e51e0491 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.169149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2178bc-b91e-4693-8a9f-3497e51e0491 admin admin] PATH_INFO: `/v3/projects/eb6a951afecc44219772f82bb09f91e3/users/6f67deff06894bffb9ac99680df14b88/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.178689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 806/1600] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:42 2022] PUT /identity/v3/projects/becec5cd5afd44848e7178d7a6c2150d/users/94ee943e661d4eb4a860fcea5db842b0/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:42.179432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82c810-a082-4d3b-b97a-7a663b109683 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.198310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2178bc-b91e-4693-8a9f-3497e51e0491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6a951afecc44219772f82bb09f91e3, user_id=6f67deff06894bffb9ac99680df14b88, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.201347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2178bc-b91e-4693-8a9f-3497e51e0491 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.217365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 795/1601] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:42 2022] PUT /identity/v3/projects/eb6a951afecc44219772f82bb09f91e3/users/6f67deff06894bffb9ac99680df14b88/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:42.218218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2178bc-b91e-4693-8a9f-3497e51e0491 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.249999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82c810-a082-4d3b-b97a-7a663b109683 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.301424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2178bc-b91e-4693-8a9f-3497e51e0491 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.307214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e3b51-e406-41a2-bfbd-5e8548040342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.308275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e3b51-e406-41a2-bfbd-5e8548040342 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.308587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e3b51-e406-41a2-bfbd-5e8548040342 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.308863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e3b51-e406-41a2-bfbd-5e8548040342 admin admin] PATH_INFO: `/v3/projects/f1064fc209674020b54fa7526cf22bf5/users/5bf265cb4af14e2d8815c42288e2a1f7/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.320659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba3461-da58-416b-b4db-e4192cd7103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.321091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba3461-da58-416b-b4db-e4192cd7103b admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.321340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba3461-da58-416b-b4db-e4192cd7103b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.321713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba3461-da58-416b-b4db-e4192cd7103b admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.322458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba3461-da58-416b-b4db-e4192cd7103b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.324255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba3461-da58-416b-b4db-e4192cd7103b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.336719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e3b51-e406-41a2-bfbd-5e8548040342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1064fc209674020b54fa7526cf22bf5, user_id=5bf265cb4af14e2d8815c42288e2a1f7, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.339070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 807/1602] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:42.339586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba3461-da58-416b-b4db-e4192cd7103b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.340408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e3b51-e406-41a2-bfbd-5e8548040342 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.351970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba3461-da58-416b-b4db-e4192cd7103b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.357417 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3f85f-987f-49c9-92e5-2c5b23fd7397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.358052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3f85f-987f-49c9-92e5-2c5b23fd7397 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.358331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3f85f-987f-49c9-92e5-2c5b23fd7397 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.358599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3f85f-987f-49c9-92e5-2c5b23fd7397 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.359279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3f85f-987f-49c9-92e5-2c5b23fd7397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.361802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3f85f-987f-49c9-92e5-2c5b23fd7397 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.367999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 796/1603] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:42 2022] PUT /identity/v3/projects/f1064fc209674020b54fa7526cf22bf5/users/5bf265cb4af14e2d8815c42288e2a1f7/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:42.368973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e3b51-e406-41a2-bfbd-5e8548040342 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.408698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 808/1604] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:42 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:42.409665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3f85f-987f-49c9-92e5-2c5b23fd7397 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.454042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e3b51-e406-41a2-bfbd-5e8548040342 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.458153 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:42.467437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:42.489201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3f85f-987f-49c9-92e5-2c5b23fd7397 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.495187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cffd05-ecf2-4126-9da1-cb4f8a5b253c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.495985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cffd05-ecf2-4126-9da1-cb4f8a5b253c admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.496266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cffd05-ecf2-4126-9da1-cb4f8a5b253c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.496609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cffd05-ecf2-4126-9da1-cb4f8a5b253c admin admin] PATH_INFO: `/v3/projects/06942912e37346eaa2310a56b84f3886/users/ecf29db866eb43fa954b0d440577f45f/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.514488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cffd05-ecf2-4126-9da1-cb4f8a5b253c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06942912e37346eaa2310a56b84f3886, user_id=ecf29db866eb43fa954b0d440577f45f, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.517907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cffd05-ecf2-4126-9da1-cb4f8a5b253c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.519631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750bcf4-d972-4020-bf5d-28519747631f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.520729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750bcf4-d972-4020-bf5d-28519747631f admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.521492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750bcf4-d972-4020-bf5d-28519747631f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.521989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750bcf4-d972-4020-bf5d-28519747631f admin admin] PATH_INFO: `/v3/projects/25fba38957464d34adbcc250cabbc8cd/users/da6eb6b18460435c93a914745d10aa7c/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.533631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 809/1605] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:42 2022] PUT /identity/v3/projects/06942912e37346eaa2310a56b84f3886/users/ecf29db866eb43fa954b0d440577f45f/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:42.534682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0ce8-82ff-4f34-bcc8-d807887bf3fa None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.534966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0ce8-82ff-4f34-bcc8-d807887bf3fa None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.535238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0ce8-82ff-4f34-bcc8-d807887bf3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.539787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750bcf4-d972-4020-bf5d-28519747631f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25fba38957464d34adbcc250cabbc8cd, user_id=da6eb6b18460435c93a914745d10aa7c, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.542545 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750bcf4-d972-4020-bf5d-28519747631f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.564591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 797/1606] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:42 2022] PUT /identity/v3/projects/25fba38957464d34adbcc250cabbc8cd/users/da6eb6b18460435c93a914745d10aa7c/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:42.565516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-234c0ce8-82ff-4f34-bcc8-d807887bf3fa None None] MFA Rules not processed for user `94ee943e661d4eb4a860fcea5db842b0`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:42.566752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-234c0ce8-82ff-4f34-bcc8-d807887bf3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:42.567079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc66868-ae45-41bb-ad87-942c0b23b8ed None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.567674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc66868-ae45-41bb-ad87-942c0b23b8ed None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.568844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc66868-ae45-41bb-ad87-942c0b23b8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.593276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-7bc66868-ae45-41bb-ad87-942c0b23b8ed None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:42.596128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7bc66868-ae45-41bb-ad87-942c0b23b8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:42.646356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 810/1607] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:42.646356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c0ce8-82ff-4f34-bcc8-d807887bf3fa None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.671055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 798/1608] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:42.675795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c812-8034-40f0-afb3-3bc4813bd12d None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.675795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c812-8034-40f0-afb3-3bc4813bd12d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.675795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c812-8034-40f0-afb3-3bc4813bd12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.694780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-9279c812-8034-40f0-afb3-3bc4813bd12d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:42.695951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9279c812-8034-40f0-afb3-3bc4813bd12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:42.707465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 799/1609] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:42.707465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279c812-8034-40f0-afb3-3bc4813bd12d None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.715798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279c812-8034-40f0-afb3-3bc4813bd12d None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.721642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c330dc-ac22-47e0-a8fc-e4c703232e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.722866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c330dc-ac22-47e0-a8fc-e4c703232e73 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.723146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c330dc-ac22-47e0-a8fc-e4c703232e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.723862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c330dc-ac22-47e0-a8fc-e4c703232e73 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.724187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c330dc-ac22-47e0-a8fc-e4c703232e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.727447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c0ce8-82ff-4f34-bcc8-d807887bf3fa None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.730189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c330dc-ac22-47e0-a8fc-e4c703232e73 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.737217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc08db7-6210-43e7-b39d-e2267fad62a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.738905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc08db7-6210-43e7-b39d-e2267fad62a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.739581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc08db7-6210-43e7-b39d-e2267fad62a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.739991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc08db7-6210-43e7-b39d-e2267fad62a9 admin admin] PATH_INFO: `/v3/projects/b620fa18ad9947a8b810be774b15e7c1/users/f336183f8533454db87e5c1c39e82e57/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.762686 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 800/1610] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:42 2022] POST /identity/v3/users => generated 300 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:42.764513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a7619-6906-4d35-a019-33e2bda35af4 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.765654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a7619-6906-4d35-a019-33e2bda35af4 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.765958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a7619-6906-4d35-a019-33e2bda35af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.772927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc08db7-6210-43e7-b39d-e2267fad62a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b620fa18ad9947a8b810be774b15e7c1, user_id=f336183f8533454db87e5c1c39e82e57, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.775712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc08db7-6210-43e7-b39d-e2267fad62a9 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.795868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-674a7619-6906-4d35-a019-33e2bda35af4 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:42.798925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-674a7619-6906-4d35-a019-33e2bda35af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:42.805585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 811/1611] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:42 2022] PUT /identity/v3/projects/b620fa18ad9947a8b810be774b15e7c1/users/f336183f8533454db87e5c1c39e82e57/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:42.805585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f633-db60-464b-98e9-27e2e12fa0fd None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.805585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f633-db60-464b-98e9-27e2e12fa0fd None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.805585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f633-db60-464b-98e9-27e2e12fa0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.843657 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5f00f633-db60-464b-98e9-27e2e12fa0fd None None] MFA Rules not processed for user `da6eb6b18460435c93a914745d10aa7c`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:42.845090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5f00f633-db60-464b-98e9-27e2e12fa0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:42.878611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:42.898506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 801/1612] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:42.898506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a7619-6906-4d35-a019-33e2bda35af4 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.911757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a7619-6906-4d35-a019-33e2bda35af4 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:42.917189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4054bd-978b-4cb0-8d60-6949f1ed8c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:42.917936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4054bd-978b-4cb0-8d60-6949f1ed8c1c admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:42.918597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4054bd-978b-4cb0-8d60-6949f1ed8c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:42.918990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4054bd-978b-4cb0-8d60-6949f1ed8c1c admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:42.920287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4054bd-978b-4cb0-8d60-6949f1ed8c1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:42.922362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4054bd-978b-4cb0-8d60-6949f1ed8c1c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:42.929295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:42.936715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 812/1613] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:42.940744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00f633-db60-464b-98e9-27e2e12fa0fd None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:42.960788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 802/1614] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:42 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:42.966148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4054bd-978b-4cb0-8d60-6949f1ed8c1c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.041578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00f633-db60-464b-98e9-27e2e12fa0fd None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.047597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6107d-bd3e-4fd1-bb79-1389fb53f918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.049540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6107d-bd3e-4fd1-bb79-1389fb53f918 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.049540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6107d-bd3e-4fd1-bb79-1389fb53f918 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.049540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6107d-bd3e-4fd1-bb79-1389fb53f918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.049871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4054bd-978b-4cb0-8d60-6949f1ed8c1c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.053802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f909af1-d3f9-4df1-b08f-91008b8d3689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.054513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f909af1-d3f9-4df1-b08f-91008b8d3689 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.054794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f909af1-d3f9-4df1-b08f-91008b8d3689 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.055110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f909af1-d3f9-4df1-b08f-91008b8d3689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.128048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6107d-bd3e-4fd1-bb79-1389fb53f918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.130006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6107d-bd3e-4fd1-bb79-1389fb53f918 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.141145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f909af1-d3f9-4df1-b08f-91008b8d3689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.142702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 813/1615] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:43.143811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f909af1-d3f9-4df1-b08f-91008b8d3689 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.144077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6107d-bd3e-4fd1-bb79-1389fb53f918 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.155989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6107d-bd3e-4fd1-bb79-1389fb53f918 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.158767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 803/1616] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:43.160689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6feb4-6306-4f27-b9c3-ba2c98b30d5b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.160984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6feb4-6306-4f27-b9c3-ba2c98b30d5b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.161256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6feb4-6306-4f27-b9c3-ba2c98b30d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.162506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf26aea-32cf-4469-a552-1e1419359849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.163276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf26aea-32cf-4469-a552-1e1419359849 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.163883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf26aea-32cf-4469-a552-1e1419359849 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.164337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf26aea-32cf-4469-a552-1e1419359849 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.165275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf26aea-32cf-4469-a552-1e1419359849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.167284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf26aea-32cf-4469-a552-1e1419359849 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.183642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 814/1617] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:43.184934 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf26aea-32cf-4469-a552-1e1419359849 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.190849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-26f6feb4-6306-4f27-b9c3-ba2c98b30d5b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:43.193705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-26f6feb4-6306-4f27-b9c3-ba2c98b30d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:43.198548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf26aea-32cf-4469-a552-1e1419359849 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.206326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca5851-34d1-41ea-93e4-93720cb79d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.206326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca5851-34d1-41ea-93e4-93720cb79d29 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.206326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca5851-34d1-41ea-93e4-93720cb79d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.206326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca5851-34d1-41ea-93e4-93720cb79d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.207463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 804/1618] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:43.208542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6feb4-6306-4f27-b9c3-ba2c98b30d5b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.224584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca5851-34d1-41ea-93e4-93720cb79d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.225897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6feb4-6306-4f27-b9c3-ba2c98b30d5b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.226857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca5851-34d1-41ea-93e4-93720cb79d29 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.239766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9cd9a-80be-4ac8-a697-cd717c46b671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.241757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9cd9a-80be-4ac8-a697-cd717c46b671 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.242042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9cd9a-80be-4ac8-a697-cd717c46b671 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.242317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9cd9a-80be-4ac8-a697-cd717c46b671 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.242961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9cd9a-80be-4ac8-a697-cd717c46b671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.245194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9cd9a-80be-4ac8-a697-cd717c46b671 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.252125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 815/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:43.256845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba719c-6363-43d1-b915-a75c108492a7 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.257323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba719c-6363-43d1-b915-a75c108492a7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.257760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba719c-6363-43d1-b915-a75c108492a7 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.266191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 816/1620] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 1 00:29:43 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:29:43.267571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba719c-6363-43d1-b915-a75c108492a7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.289255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 805/1621] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:43 2022] POST /identity/v3/users => generated 293 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:43.290499 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9cd9a-80be-4ac8-a697-cd717c46b671 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.292615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1cb9cd9a-80be-4ac8-a697-cd717c46b671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:43.294209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba719c-6363-43d1-b915-a75c108492a7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.300741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a11d91-d4a4-4527-bc4a-4c131fa7dbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.301756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11d91-d4a4-4527-bc4a-4c131fa7dbe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.302043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11d91-d4a4-4527-bc4a-4c131fa7dbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.302323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11d91-d4a4-4527-bc4a-4c131fa7dbe8 admin admin] PATH_INFO: `/v3/projects/a924b839b88347a7b299a5c8c19919ca/users/eb65cf71936b45f3a37390400b037bcb/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.311224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9cd9a-80be-4ac8-a697-cd717c46b671 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.317453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640e28a-1ccc-429b-ad09-4f3089c0db01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.317453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640e28a-1ccc-429b-ad09-4f3089c0db01 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.317758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640e28a-1ccc-429b-ad09-4f3089c0db01 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.318076 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0640e28a-1ccc-429b-ad09-4f3089c0db01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.323477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a11d91-d4a4-4527-bc4a-4c131fa7dbe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a924b839b88347a7b299a5c8c19919ca, user_id=eb65cf71936b45f3a37390400b037bcb, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.327488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a11d91-d4a4-4527-bc4a-4c131fa7dbe8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.328915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640e28a-1ccc-429b-ad09-4f3089c0db01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.330944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0640e28a-1ccc-429b-ad09-4f3089c0db01 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.349495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 817/1622] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:43 2022] PUT /identity/v3/projects/a924b839b88347a7b299a5c8c19919ca/users/eb65cf71936b45f3a37390400b037bcb/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:43.351006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b396c2-cc20-4c16-9e24-00515041f09b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.351384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b396c2-cc20-4c16-9e24-00515041f09b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.351810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b396c2-cc20-4c16-9e24-00515041f09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.358453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 806/1623] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:43.360339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640e28a-1ccc-429b-ad09-4f3089c0db01 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.393987 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0640e28a-1ccc-429b-ad09-4f3089c0db01 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.411115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a3b396c2-cc20-4c16-9e24-00515041f09b None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:43.412525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a3b396c2-cc20-4c16-9e24-00515041f09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:43.426303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:43.543251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:43.555347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fad22-6286-419d-ac4e-469e5d5355e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.556206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fad22-6286-419d-ac4e-469e5d5355e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.556513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fad22-6286-419d-ac4e-469e5d5355e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.556990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fad22-6286-419d-ac4e-469e5d5355e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.557704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fad22-6286-419d-ac4e-469e5d5355e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.559237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 818/1624] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 1 00:29:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:43.560114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b396c2-cc20-4c16-9e24-00515041f09b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.570135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fad22-6286-419d-ac4e-469e5d5355e8 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.576240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b396c2-cc20-4c16-9e24-00515041f09b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.583608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965763bc-b4da-4bbf-88b9-43f44cff6112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.584794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965763bc-b4da-4bbf-88b9-43f44cff6112 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.585636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965763bc-b4da-4bbf-88b9-43f44cff6112 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.586005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965763bc-b4da-4bbf-88b9-43f44cff6112 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.587116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965763bc-b4da-4bbf-88b9-43f44cff6112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.589234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 807/1625] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:43.589629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965763bc-b4da-4bbf-88b9-43f44cff6112 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.590232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fad22-6286-419d-ac4e-469e5d5355e8 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.605426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 819/1626] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:43.606366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965763bc-b4da-4bbf-88b9-43f44cff6112 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.607053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fad22-6286-419d-ac4e-469e5d5355e8 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.614984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428d431-dedc-449d-91e8-f76ad22153dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.615765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428d431-dedc-449d-91e8-f76ad22153dc service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.616152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428d431-dedc-449d-91e8-f76ad22153dc service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.617007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428d431-dedc-449d-91e8-f76ad22153dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.617385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965763bc-b4da-4bbf-88b9-43f44cff6112 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.622948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524751e-8ef8-45a4-b691-ca209a4a805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.623662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524751e-8ef8-45a4-b691-ca209a4a805f admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.624518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524751e-8ef8-45a4-b691-ca209a4a805f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.624812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524751e-8ef8-45a4-b691-ca209a4a805f admin admin] PATH_INFO: `/v3/projects/8e136a2576c3475481e580f8debfc295/users/fc159728817947a7a68d669f68d3e36f/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.643969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524751e-8ef8-45a4-b691-ca209a4a805f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e136a2576c3475481e580f8debfc295, user_id=fc159728817947a7a68d669f68d3e36f, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.647837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524751e-8ef8-45a4-b691-ca209a4a805f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.664873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 820/1627] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:43 2022] PUT /identity/v3/projects/8e136a2576c3475481e580f8debfc295/users/fc159728817947a7a68d669f68d3e36f/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:43.665711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524751e-8ef8-45a4-b691-ca209a4a805f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.726728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428d431-dedc-449d-91e8-f76ad22153dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.729346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428d431-dedc-449d-91e8-f76ad22153dc service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.761906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 808/1628] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:43.766457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428d431-dedc-449d-91e8-f76ad22153dc service nova] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.777104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524751e-8ef8-45a4-b691-ca209a4a805f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.788671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea948d7-e1da-43df-8e72-bbe9028e7161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.790524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea948d7-e1da-43df-8e72-bbe9028e7161 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.791030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea948d7-e1da-43df-8e72-bbe9028e7161 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.791480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea948d7-e1da-43df-8e72-bbe9028e7161 admin admin] PATH_INFO: `/v3/projects/a924b839b88347a7b299a5c8c19919ca/users/eb65cf71936b45f3a37390400b037bcb/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.796444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea948d7-e1da-43df-8e72-bbe9028e7161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a924b839b88347a7b299a5c8c19919ca, user_id=eb65cf71936b45f3a37390400b037bcb, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.799690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428d431-dedc-449d-91e8-f76ad22153dc service nova] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.803238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea948d7-e1da-43df-8e72-bbe9028e7161 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.814492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba3fd-3977-4950-9e49-fc299c06900c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:43.815322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba3fd-3977-4950-9e49-fc299c06900c admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.815604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba3fd-3977-4950-9e49-fc299c06900c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.815890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ba3fd-3977-4950-9e49-fc299c06900c admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.816575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ba3fd-3977-4950-9e49-fc299c06900c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:43.819509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ba3fd-3977-4950-9e49-fc299c06900c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:43.901031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 821/1629] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:43 2022] PUT /identity/v3/projects/a924b839b88347a7b299a5c8c19919ca/users/eb65cf71936b45f3a37390400b037bcb/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:43.935769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 809/1630] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:43.936774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76b579-e57e-455d-9d52-4413d799e252 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:43.936774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76b579-e57e-455d-9d52-4413d799e252 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:43.936774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76b579-e57e-455d-9d52-4413d799e252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:43.937241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba3fd-3977-4950-9e49-fc299c06900c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:43.948680 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ba3fd-3977-4950-9e49-fc299c06900c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:43.971938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-7c76b579-e57e-455d-9d52-4413d799e252 None None] MFA Rules not processed for user `eb65cf71936b45f3a37390400b037bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:43.973712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7c76b579-e57e-455d-9d52-4413d799e252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:44.113523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692770f9-a3ae-4533-9bed-9df05e24a1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:44.115394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692770f9-a3ae-4533-9bed-9df05e24a1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:44.115685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692770f9-a3ae-4533-9bed-9df05e24a1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:44.115960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692770f9-a3ae-4533-9bed-9df05e24a1a6 admin admin] PATH_INFO: `/v3/projects/8e136a2576c3475481e580f8debfc295/users/fc159728817947a7a68d669f68d3e36f/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:44.123179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692770f9-a3ae-4533-9bed-9df05e24a1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e136a2576c3475481e580f8debfc295, user_id=fc159728817947a7a68d669f68d3e36f, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:44.130848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692770f9-a3ae-4533-9bed-9df05e24a1a6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:44.145066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:44.169680 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 810/1631] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:43 2022] PUT /identity/v3/projects/8e136a2576c3475481e580f8debfc295/users/fc159728817947a7a68d669f68d3e36f/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:44.171640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030a8be-ecb5-427a-a7ee-0ce852334912 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:44.171925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030a8be-ecb5-427a-a7ee-0ce852334912 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:44.176483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030a8be-ecb5-427a-a7ee-0ce852334912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:44.178687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 822/1632] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:44.186515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76b579-e57e-455d-9d52-4413d799e252 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:44.236624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2030a8be-ecb5-427a-a7ee-0ce852334912 None None] MFA Rules not processed for user `fc159728817947a7a68d669f68d3e36f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:44.238192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2030a8be-ecb5-427a-a7ee-0ce852334912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:44.307396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76b579-e57e-455d-9d52-4413d799e252 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:44.374541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205c9d-4840-4c5f-9e40-e883adc8833f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:44.375477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205c9d-4840-4c5f-9e40-e883adc8833f service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:44.375766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205c9d-4840-4c5f-9e40-e883adc8833f service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:44.376101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205c9d-4840-4c5f-9e40-e883adc8833f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:44.378533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:44.386325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 811/1633] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:44.389876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4700fd5-5b1e-4e99-b36b-d8794d52d14e None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:44.390561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4700fd5-5b1e-4e99-b36b-d8794d52d14e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:44.390856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4700fd5-5b1e-4e99-b36b-d8794d52d14e None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:44.393813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 812/1634] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 1 00:29:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:29:44.416461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada95d8-47f6-4c30-a73a-d4036bd0625f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:44.419073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada95d8-47f6-4c30-a73a-d4036bd0625f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:44.420391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada95d8-47f6-4c30-a73a-d4036bd0625f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:44.477971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-7ada95d8-47f6-4c30-a73a-d4036bd0625f None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:44.482552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7ada95d8-47f6-4c30-a73a-d4036bd0625f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:44.495709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39205c9d-4840-4c5f-9e40-e883adc8833f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:44.497665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39205c9d-4840-4c5f-9e40-e883adc8833f service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:44.530275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 823/1635] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:29:44.531866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205c9d-4840-4c5f-9e40-e883adc8833f service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:44.579157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205c9d-4840-4c5f-9e40-e883adc8833f service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:44.594655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d93ed-2fb3-4f52-9134-e326d8d971f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:44.598270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d93ed-2fb3-4f52-9134-e326d8d971f9 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:44.599095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d93ed-2fb3-4f52-9134-e326d8d971f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:44.599682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d93ed-2fb3-4f52-9134-e326d8d971f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:44.630320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d93ed-2fb3-4f52-9134-e326d8d971f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:44.632326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d93ed-2fb3-4f52-9134-e326d8d971f9 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:44.663419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 813/1636] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:29:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:29:44.664425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada95d8-47f6-4c30-a73a-d4036bd0625f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:44.676743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 824/1637] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:44.679179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d93ed-2fb3-4f52-9134-e326d8d971f9 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:44.703916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d93ed-2fb3-4f52-9134-e326d8d971f9 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:44.726270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bab6fb-9efb-4986-adac-1c7bd4dd2072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:44.729203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bab6fb-9efb-4986-adac-1c7bd4dd2072 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:44.729868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bab6fb-9efb-4986-adac-1c7bd4dd2072 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:44.730148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bab6fb-9efb-4986-adac-1c7bd4dd2072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:44.778555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bab6fb-9efb-4986-adac-1c7bd4dd2072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:44.778555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bab6fb-9efb-4986-adac-1c7bd4dd2072 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:44.817620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 825/1638] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:44.835472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada95d8-47f6-4c30-a73a-d4036bd0625f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:44.842706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:44.843785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:44.843785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:44.844283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] PATH_INFO: `/v3/users/67dd50d6270248f298233ee15d2cd23a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:44.846291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] RBAC: Authorizing `identity:delete_user(user_id=67dd50d6270248f298233ee15d2cd23a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:44.853859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:44.959601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67dd50d6270248f298233ee15d2cd23a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:44.960870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67dd50d6270248f298233ee15d2cd23a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:44.990518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67dd50d6270248f298233ee15d2cd23a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:45.007098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67dd50d6270248f298233ee15d2cd23a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:45.022791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0c60e3d2-6b06-40d2-91ba-039df18a351b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67dd50d6270248f298233ee15d2cd23a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:45.034630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 814/1639] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:29:44 2022] DELETE /identity/v3/users/67dd50d6270248f298233ee15d2cd23a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:45.040892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7074252-f479-4a30-bb6f-0a8fa8a2f1e2 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:45.041506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7074252-f479-4a30-bb6f-0a8fa8a2f1e2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:45.041985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7074252-f479-4a30-bb6f-0a8fa8a2f1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:45.094105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906a1f3-0155-4be9-8de7-11f5c2a3efa1 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:45.094390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906a1f3-0155-4be9-8de7-11f5c2a3efa1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:45.094664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906a1f3-0155-4be9-8de7-11f5c2a3efa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:45.116941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e7074252-f479-4a30-bb6f-0a8fa8a2f1e2 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:45.119145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e7074252-f479-4a30-bb6f-0a8fa8a2f1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:45.149387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:45.174097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:45.197018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e906a1f3-0155-4be9-8de7-11f5c2a3efa1 None None] MFA Rules not processed for user `f336183f8533454db87e5c1c39e82e57`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:45.202587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e906a1f3-0155-4be9-8de7-11f5c2a3efa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:45.245159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 826/1640] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:45.277194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7074252-f479-4a30-bb6f-0a8fa8a2f1e2 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:45.277194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e7074252-f479-4a30-bb6f-0a8fa8a2f1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:45.407050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 815/1641] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:45 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:45.409990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906a1f3-0155-4be9-8de7-11f5c2a3efa1 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:45.412999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e906a1f3-0155-4be9-8de7-11f5c2a3efa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:45.442385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906a1f3-0155-4be9-8de7-11f5c2a3efa1 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:45.457921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a0b66-e6bd-4b0e-a83b-85a6349945eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:45.458451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a0b66-e6bd-4b0e-a83b-85a6349945eb admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:45.459227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a0b66-e6bd-4b0e-a83b-85a6349945eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:45.459521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a0b66-e6bd-4b0e-a83b-85a6349945eb admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:45.460902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a0b66-e6bd-4b0e-a83b-85a6349945eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:45.466676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a0b66-e6bd-4b0e-a83b-85a6349945eb admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:45.505326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7074252-f479-4a30-bb6f-0a8fa8a2f1e2 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:45.536773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 816/1642] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:45 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:29:45.538092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a0b66-e6bd-4b0e-a83b-85a6349945eb admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:45.545344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d9ecf-5d43-4d46-8d1b-8720ba239f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:45.546140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d9ecf-5d43-4d46-8d1b-8720ba239f1b service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:45.546416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d9ecf-5d43-4d46-8d1b-8720ba239f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:45.546691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d9ecf-5d43-4d46-8d1b-8720ba239f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:45.573931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d9ecf-5d43-4d46-8d1b-8720ba239f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:45.575853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d9ecf-5d43-4d46-8d1b-8720ba239f1b service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:45.603442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 827/1643] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:29:45.621238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a0b66-e6bd-4b0e-a83b-85a6349945eb admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:45.628722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29a89d-fd83-4bc8-810e-b2e26715d871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:45.629484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29a89d-fd83-4bc8-810e-b2e26715d871 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:45.629773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29a89d-fd83-4bc8-810e-b2e26715d871 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:45.630600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29a89d-fd83-4bc8-810e-b2e26715d871 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:45.631381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29a89d-fd83-4bc8-810e-b2e26715d871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:45.636635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29a89d-fd83-4bc8-810e-b2e26715d871 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:45.637917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d9ecf-5d43-4d46-8d1b-8720ba239f1b service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:45.640120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e95d9ecf-5d43-4d46-8d1b-8720ba239f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:45.663832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d9ecf-5d43-4d46-8d1b-8720ba239f1b service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:45.671155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 817/1644] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:45 2022] POST /identity/v3/users => generated 309 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:29:45.672225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29a89d-fd83-4bc8-810e-b2e26715d871 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:45.675285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:45.675931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:45.676207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:45.676477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] PATH_INFO: `/v3/users/94ee943e661d4eb4a860fcea5db842b0` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:45.680169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ee943e661d4eb4a860fcea5db842b0)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:45.685082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: WARNING py.warnings [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:29:45.685082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: warnings.warn(msg) Sep 01 00:29:45.685082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: Sep 01 00:29:45.686349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:45.687548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29a89d-fd83-4bc8-810e-b2e26715d871 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:45.696525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde08e26-201a-46fe-b876-b1e4901c8d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:45.697357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde08e26-201a-46fe-b876-b1e4901c8d7c admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:45.697808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde08e26-201a-46fe-b876-b1e4901c8d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:45.698241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde08e26-201a-46fe-b876-b1e4901c8d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:45.699239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde08e26-201a-46fe-b876-b1e4901c8d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:45.701605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde08e26-201a-46fe-b876-b1e4901c8d7c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:45.715944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 818/1645] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:45.719844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde08e26-201a-46fe-b876-b1e4901c8d7c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:45.738328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde08e26-201a-46fe-b876-b1e4901c8d7c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:45.745916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a45fc-8750-405f-849a-9edcdffba94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:45.747466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a45fc-8750-405f-849a-9edcdffba94e admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:45.748023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a45fc-8750-405f-849a-9edcdffba94e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:45.748584 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a45fc-8750-405f-849a-9edcdffba94e admin admin] PATH_INFO: `/v3/projects/25cc8a74338e49e7b7332b34c30c1c43/users/ee1a8c58a0ec4a69835f67c2336eb1e6/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:45.775886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a45fc-8750-405f-849a-9edcdffba94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25cc8a74338e49e7b7332b34c30c1c43, user_id=ee1a8c58a0ec4a69835f67c2336eb1e6, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:45.779151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a45fc-8750-405f-849a-9edcdffba94e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:45.804934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 819/1646] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:45 2022] PUT /identity/v3/projects/25cc8a74338e49e7b7332b34c30c1c43/users/ee1a8c58a0ec4a69835f67c2336eb1e6/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:45.806903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a45fc-8750-405f-849a-9edcdffba94e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:45.857038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ee943e661d4eb4a860fcea5db842b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:45.858441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ee943e661d4eb4a860fcea5db842b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:45.868254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ee943e661d4eb4a860fcea5db842b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:45.872079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:45.886493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ee943e661d4eb4a860fcea5db842b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:45.887728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a7248848-ed85-478b-9d07-7d29725c77cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ee943e661d4eb4a860fcea5db842b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:45.938386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a45fc-8750-405f-849a-9edcdffba94e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:45.949340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 828/1647] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:29:45 2022] DELETE /identity/v3/users/94ee943e661d4eb4a860fcea5db842b0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:45.951434 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30636b0-1e12-497b-abcc-1da3bf77653f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:45.952810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30636b0-1e12-497b-abcc-1da3bf77653f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:45.953823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30636b0-1e12-497b-abcc-1da3bf77653f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:46.010075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-c30636b0-1e12-497b-abcc-1da3bf77653f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:46.020269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c30636b0-1e12-497b-abcc-1da3bf77653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:46.126424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaed35f-503f-4618-bb7b-45fec46784bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:46.128715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaed35f-503f-4618-bb7b-45fec46784bd admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:46.129156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaed35f-503f-4618-bb7b-45fec46784bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:46.129610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaed35f-503f-4618-bb7b-45fec46784bd admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:46.131474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaed35f-503f-4618-bb7b-45fec46784bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:46.135181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaed35f-503f-4618-bb7b-45fec46784bd admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:46.162919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:46.168589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 820/1648] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:46.170660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaed35f-503f-4618-bb7b-45fec46784bd admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:46.172718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4eaed35f-503f-4618-bb7b-45fec46784bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:46.194057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaed35f-503f-4618-bb7b-45fec46784bd admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:46.201312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ebd45-0225-4fe5-8f19-1f2a598a4434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:46.202165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ebd45-0225-4fe5-8f19-1f2a598a4434 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:46.202649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ebd45-0225-4fe5-8f19-1f2a598a4434 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:46.203044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ebd45-0225-4fe5-8f19-1f2a598a4434 admin admin] PATH_INFO: `/v3/projects/25cc8a74338e49e7b7332b34c30c1c43/users/ee1a8c58a0ec4a69835f67c2336eb1e6/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:46.205395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ebd45-0225-4fe5-8f19-1f2a598a4434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25cc8a74338e49e7b7332b34c30c1c43, user_id=ee1a8c58a0ec4a69835f67c2336eb1e6, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:46.210750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ebd45-0225-4fe5-8f19-1f2a598a4434 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:46.226400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 821/1649] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:46 2022] PUT /identity/v3/projects/25cc8a74338e49e7b7332b34c30c1c43/users/ee1a8c58a0ec4a69835f67c2336eb1e6/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:29:46.231917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 829/1650] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:46.324877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ebd45-0225-4fe5-8f19-1f2a598a4434 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:46.326786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1a8ebd45-0225-4fe5-8f19-1f2a598a4434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:46.475665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b98e5-d394-4b54-8cf9-837ca632b39a None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:46.476449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b98e5-d394-4b54-8cf9-837ca632b39a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:46.477079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b98e5-d394-4b54-8cf9-837ca632b39a None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:46.479469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 830/1651] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 1 00:29:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:29:46.484145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ebd45-0225-4fe5-8f19-1f2a598a4434 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:46.491060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe9fe2-84a1-4a40-aa39-44de0fe785c9 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:46.492027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe9fe2-84a1-4a40-aa39-44de0fe785c9 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:46.492642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe9fe2-84a1-4a40-aa39-44de0fe785c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:46.492997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291b5f1-12fe-4450-9936-c26e36d608ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:46.494980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291b5f1-12fe-4450-9936-c26e36d608ae service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:46.495855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291b5f1-12fe-4450-9936-c26e36d608ae service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:46.496143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291b5f1-12fe-4450-9936-c26e36d608ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:46.537016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-aefe9fe2-84a1-4a40-aa39-44de0fe785c9 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:46.539457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-aefe9fe2-84a1-4a40-aa39-44de0fe785c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:46.640077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 831/1652] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 1 00:29:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:46.642734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5e7a-5f05-47a5-ab6b-86349d100e01 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:46.643420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5e7a-5f05-47a5-ab6b-86349d100e01 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:46.643907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5e7a-5f05-47a5-ab6b-86349d100e01 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:46.645871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 832/1653] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 1 00:29:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:29:46.653437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ceb7f-df4d-4128-9d56-bf871a9cb6d9 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:46.654094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ceb7f-df4d-4128-9d56-bf871a9cb6d9 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:46.654771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ceb7f-df4d-4128-9d56-bf871a9cb6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:46.662415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291b5f1-12fe-4450-9936-c26e36d608ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:46.666956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291b5f1-12fe-4450-9936-c26e36d608ae service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:46.680562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6a3ceb7f-df4d-4128-9d56-bf871a9cb6d9 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:46.682162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6a3ceb7f-df4d-4128-9d56-bf871a9cb6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:46.691583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 822/1654] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:46.713183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 833/1655] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:29:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:46.713553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291b5f1-12fe-4450-9936-c26e36d608ae service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:46.714992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6291b5f1-12fe-4450-9936-c26e36d608ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:46.738754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:46.829984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291b5f1-12fe-4450-9936-c26e36d608ae service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:46.835570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b6e35-32a6-48e9-b293-d278b6c3ab34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:46.836840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b6e35-32a6-48e9-b293-d278b6c3ab34 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:46.837702 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b6e35-32a6-48e9-b293-d278b6c3ab34 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:46.838188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b6e35-32a6-48e9-b293-d278b6c3ab34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:46.868906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b6e35-32a6-48e9-b293-d278b6c3ab34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:46.872959 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b6e35-32a6-48e9-b293-d278b6c3ab34 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:46.920368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 823/1656] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:29:47.685001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ceb7f-df4d-4128-9d56-bf871a9cb6d9 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:47.690379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6a3ceb7f-df4d-4128-9d56-bf871a9cb6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:47.714890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ceb7f-df4d-4128-9d56-bf871a9cb6d9 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:47.721915 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:47.731749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:47.736185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:47.738336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] PATH_INFO: `/v3/projects/cb964432f4ad4e0c8d38d9449ebfaf41` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:47.740834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb964432f4ad4e0c8d38d9449ebfaf41)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:47.744348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:47.841126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb964432f4ad4e0c8d38d9449ebfaf41', 'request_id': 'req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:47.849302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb964432f4ad4e0c8d38d9449ebfaf41', 'request_id': 'req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:47.864438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb964432f4ad4e0c8d38d9449ebfaf41', 'request_id': 'req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:47.898022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] The token cache is being invalidate because project cb964432f4ad4e0c8d38d9449ebfaf41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:29:47.898252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:47.900010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:47.902988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 834/1657] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:29:47 2022] DELETE /identity/v3/projects/cb964432f4ad4e0c8d38d9449ebfaf41 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:47.916127 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b6e35-32a6-48e9-b293-d278b6c3ab34 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:47.933611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b6e35-32a6-48e9-b293-d278b6c3ab34 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:48.051424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:48.053278 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:48.054356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:48.054779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] PATH_INFO: `/v3/users/eb65cf71936b45f3a37390400b037bcb` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:48.056179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb65cf71936b45f3a37390400b037bcb)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:48.058497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:48.189812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb65cf71936b45f3a37390400b037bcb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.191903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb65cf71936b45f3a37390400b037bcb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.231059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb65cf71936b45f3a37390400b037bcb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.271166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb65cf71936b45f3a37390400b037bcb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.297309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb65cf71936b45f3a37390400b037bcb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.309771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 824/1658] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:29:47 2022] DELETE /identity/v3/users/eb65cf71936b45f3a37390400b037bcb => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:48.592161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:48.593670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:48.615141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8046dd-6ec8-433f-abd8-bbc0bbff59ad admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:48.773390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:48.796160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:48.797211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:48.799303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:48.799794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] PATH_INFO: `/v3/users/da6eb6b18460435c93a914745d10aa7c` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:48.801324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da6eb6b18460435c93a914745d10aa7c)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:48.804772 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:48.857408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:48.858971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:48.881699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b0ae0-8361-4175-a66b-ee771a6e5030 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:48.893144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da6eb6b18460435c93a914745d10aa7c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.894118 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da6eb6b18460435c93a914745d10aa7c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.901863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:48.902533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:48.902829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:48.903107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] PATH_INFO: `/v3/projects/becec5cd5afd44848e7178d7a6c2150d` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:48.905485 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=becec5cd5afd44848e7178d7a6c2150d)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:48.906034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: WARNING py.warnings [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 00:29:48.906034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: warnings.warn(msg) Sep 01 00:29:48.906034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: Sep 01 00:29:48.908398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:48.925514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da6eb6b18460435c93a914745d10aa7c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.957402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da6eb6b18460435c93a914745d10aa7c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.961608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-624947a4-5a2f-4904-bebf-d403feb5c464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da6eb6b18460435c93a914745d10aa7c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:48.995744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 835/1659] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:29:48 2022] DELETE /identity/v3/users/da6eb6b18460435c93a914745d10aa7c => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:48.998583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:49.004992 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b3bfb-bde9-497e-8ec1-f141a98f4881 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:49.007231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b3bfb-bde9-497e-8ec1-f141a98f4881 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:49.007878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b3bfb-bde9-497e-8ec1-f141a98f4881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:49.029873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'becec5cd5afd44848e7178d7a6c2150d', 'request_id': 'req-48449bce-91f7-4285-a0d9-1a4cc5b69407'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.042694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'becec5cd5afd44848e7178d7a6c2150d', 'request_id': 'req-48449bce-91f7-4285-a0d9-1a4cc5b69407'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.059370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'becec5cd5afd44848e7178d7a6c2150d', 'request_id': 'req-48449bce-91f7-4285-a0d9-1a4cc5b69407'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.077900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-504b3bfb-bde9-497e-8ec1-f141a98f4881 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:49.082869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-504b3bfb-bde9-497e-8ec1-f141a98f4881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:49.084948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] The token cache is being invalidate because project becec5cd5afd44848e7178d7a6c2150d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:29:49.085252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.086004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.091164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 825/1660] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:29:48 2022] DELETE /identity/v3/projects/becec5cd5afd44848e7178d7a6c2150d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:49.091615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:49.093148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:49.268231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48449bce-91f7-4285-a0d9-1a4cc5b69407 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:49.342983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 836/1661] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:49.373873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b3bfb-bde9-497e-8ec1-f141a98f4881 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:49.375693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-504b3bfb-bde9-497e-8ec1-f141a98f4881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:49.467299 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:49.468632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:49.468918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:49.469196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] PATH_INFO: `/v3/users/fc159728817947a7a68d669f68d3e36f` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:49.475240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc159728817947a7a68d669f68d3e36f)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:49.477271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:49.551251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b3bfb-bde9-497e-8ec1-f141a98f4881 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:49.566092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90f73c-b134-4d85-8a9e-5c4e024755fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:49.566729 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f73c-b134-4d85-8a9e-5c4e024755fd service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:49.567023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f73c-b134-4d85-8a9e-5c4e024755fd service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:49.567285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90f73c-b134-4d85-8a9e-5c4e024755fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:49.589352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90f73c-b134-4d85-8a9e-5c4e024755fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:49.593322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90f73c-b134-4d85-8a9e-5c4e024755fd service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:49.618984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc159728817947a7a68d669f68d3e36f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.619805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc159728817947a7a68d669f68d3e36f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.625599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 837/1662] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:29:49.654699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc159728817947a7a68d669f68d3e36f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.661862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:49.668819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc159728817947a7a68d669f68d3e36f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.681871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc159728817947a7a68d669f68d3e36f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:49.696697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 826/1663] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:29:49 2022] DELETE /identity/v3/users/fc159728817947a7a68d669f68d3e36f => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:50.417332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230b9cd-17a1-4bb6-bb0c-8b8c784346e6 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:50.417909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230b9cd-17a1-4bb6-bb0c-8b8c784346e6 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:50.417909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c230b9cd-17a1-4bb6-bb0c-8b8c784346e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:50.479358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-c230b9cd-17a1-4bb6-bb0c-8b8c784346e6 None None] MFA Rules not processed for user `ee1a8c58a0ec4a69835f67c2336eb1e6`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:50.480543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c230b9cd-17a1-4bb6-bb0c-8b8c784346e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:50.496290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:50.614080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:50.615516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:50.722440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6df985-e3cd-41d4-a126-81acf1aee4d9 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:50.728449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:50.729628 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:50.730270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:50.730854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] PATH_INFO: `/v3/projects/a924b839b88347a7b299a5c8c19919ca` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:50.732750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a924b839b88347a7b299a5c8c19919ca)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:50.735618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:50.764737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 838/1664] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:50 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:50.816307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a924b839b88347a7b299a5c8c19919ca', 'request_id': 'req-9857d773-116e-4d48-8325-eb30556e251c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:50.839939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a924b839b88347a7b299a5c8c19919ca', 'request_id': 'req-9857d773-116e-4d48-8325-eb30556e251c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:50.863273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a924b839b88347a7b299a5c8c19919ca', 'request_id': 'req-9857d773-116e-4d48-8325-eb30556e251c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:50.888543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] The token cache is being invalidate because project a924b839b88347a7b299a5c8c19919ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:29:50.888746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:50.891561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9857d773-116e-4d48-8325-eb30556e251c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:50.897134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 827/1665] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:29:50 2022] DELETE /identity/v3/projects/a924b839b88347a7b299a5c8c19919ca => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:50.899842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2358f-9d75-4714-8ccf-cf54504b72fd None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:50.900440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2358f-9d75-4714-8ccf-cf54504b72fd None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:50.900886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2358f-9d75-4714-8ccf-cf54504b72fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:50.935420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-62a2358f-9d75-4714-8ccf-cf54504b72fd None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:50.937031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-62a2358f-9d75-4714-8ccf-cf54504b72fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:51.118597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 839/1666] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:51.121182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702863f4-1bf6-4da8-b16d-b84c142cf488 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:51.121478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702863f4-1bf6-4da8-b16d-b84c142cf488 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:51.121758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702863f4-1bf6-4da8-b16d-b84c142cf488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:51.148038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-702863f4-1bf6-4da8-b16d-b84c142cf488 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:51.150334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-702863f4-1bf6-4da8-b16d-b84c142cf488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:51.180347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 828/1667] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:51.183427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2358f-9d75-4714-8ccf-cf54504b72fd None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:51.199326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2358f-9d75-4714-8ccf-cf54504b72fd None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:51.212715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189cb21-7d78-4a69-bf84-7d0bc283fbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:51.214199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189cb21-7d78-4a69-bf84-7d0bc283fbb1 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:51.215018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189cb21-7d78-4a69-bf84-7d0bc283fbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:51.215539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189cb21-7d78-4a69-bf84-7d0bc283fbb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:51.216462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189cb21-7d78-4a69-bf84-7d0bc283fbb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:51.217009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189cb21-7d78-4a69-bf84-7d0bc283fbb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:51.218865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189cb21-7d78-4a69-bf84-7d0bc283fbb1 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:51.251113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 840/1668] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:51.260775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702863f4-1bf6-4da8-b16d-b84c142cf488 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:51.287089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702863f4-1bf6-4da8-b16d-b84c142cf488 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:51.311183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9781d-43a9-404e-a3d8-3b6b3888c9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:51.311974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9781d-43a9-404e-a3d8-3b6b3888c9bd admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:51.312547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9781d-43a9-404e-a3d8-3b6b3888c9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:51.313612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9781d-43a9-404e-a3d8-3b6b3888c9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:51.318350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9781d-43a9-404e-a3d8-3b6b3888c9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:51.324250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9781d-43a9-404e-a3d8-3b6b3888c9bd admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:51.455866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 829/1669] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:51 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:51.457410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189cb21-7d78-4a69-bf84-7d0bc283fbb1 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:51.474916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189cb21-7d78-4a69-bf84-7d0bc283fbb1 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:51.534333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9781d-43a9-404e-a3d8-3b6b3888c9bd admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:51.536969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-54c9781d-43a9-404e-a3d8-3b6b3888c9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:51.754524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b957722-9654-410d-940f-c31ed969668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:51.756560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b957722-9654-410d-940f-c31ed969668d admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:51.757146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b957722-9654-410d-940f-c31ed969668d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:51.758323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b957722-9654-410d-940f-c31ed969668d admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:51.759160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b957722-9654-410d-940f-c31ed969668d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:51.763984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:51.773219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b957722-9654-410d-940f-c31ed969668d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:51.834511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9781d-43a9-404e-a3d8-3b6b3888c9bd admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:51.851520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 841/1670] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:51 2022] POST /identity/v3/users => generated 290 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:51.853526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b957722-9654-410d-940f-c31ed969668d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:51.873355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:51.874028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:51.874301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:51.874651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] PATH_INFO: `/v3/projects/25fba38957464d34adbcc250cabbc8cd` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:51.876441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25fba38957464d34adbcc250cabbc8cd)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:51.878898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b957722-9654-410d-940f-c31ed969668d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:51.886786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:51.887693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102ef49-2a10-4cf3-9986-361ea6e5ddb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:51.903342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102ef49-2a10-4cf3-9986-361ea6e5ddb9 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:51.904079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102ef49-2a10-4cf3-9986-361ea6e5ddb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:51.904798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102ef49-2a10-4cf3-9986-361ea6e5ddb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:51.905893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102ef49-2a10-4cf3-9986-361ea6e5ddb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:51.911369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102ef49-2a10-4cf3-9986-361ea6e5ddb9 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:51.931054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 842/1671] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:51.932583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102ef49-2a10-4cf3-9986-361ea6e5ddb9 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:51.950038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102ef49-2a10-4cf3-9986-361ea6e5ddb9 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:51.977847 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768edb49-7905-4248-86d2-640a09e971ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:51.978575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768edb49-7905-4248-86d2-640a09e971ff admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:51.978862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768edb49-7905-4248-86d2-640a09e971ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:51.979621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768edb49-7905-4248-86d2-640a09e971ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:51.980334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768edb49-7905-4248-86d2-640a09e971ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:51.982626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768edb49-7905-4248-86d2-640a09e971ff admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:51.986089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:52.028157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25fba38957464d34adbcc250cabbc8cd', 'request_id': 'req-5d9da1a7-455c-4633-bb59-67de98d5d130'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.029344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 843/1672] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:52.031610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768edb49-7905-4248-86d2-640a09e971ff admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:52.038431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-768edb49-7905-4248-86d2-640a09e971ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:52.047283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25fba38957464d34adbcc250cabbc8cd', 'request_id': 'req-5d9da1a7-455c-4633-bb59-67de98d5d130'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.061038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768edb49-7905-4248-86d2-640a09e971ff admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:52.072872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25fba38957464d34adbcc250cabbc8cd', 'request_id': 'req-5d9da1a7-455c-4633-bb59-67de98d5d130'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.097600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] The token cache is being invalidate because project 25fba38957464d34adbcc250cabbc8cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:29:52.097893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.101355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.105803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 830/1673] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:29:51 2022] DELETE /identity/v3/projects/25fba38957464d34adbcc250cabbc8cd => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:52.109850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:52.119404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:52.206885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:52.207541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.208130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.208415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] PATH_INFO: `/v3/projects/8e136a2576c3475481e580f8debfc295` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.214324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e136a2576c3475481e580f8debfc295)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:52.219534 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:52.306278 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9da1a7-455c-4633-bb59-67de98d5d130 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:52.322152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f351f61-8ff5-4a6e-af41-e9c911b94a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:52.322834 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f351f61-8ff5-4a6e-af41-e9c911b94a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.323111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f351f61-8ff5-4a6e-af41-e9c911b94a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.323379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f351f61-8ff5-4a6e-af41-e9c911b94a1f admin admin] PATH_INFO: `/v3/projects/ddd7e3061b8d42d6b547bfc99a119f0e/users/c279954e82e14184b669a3bcedf77ca9/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.356783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f351f61-8ff5-4a6e-af41-e9c911b94a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd7e3061b8d42d6b547bfc99a119f0e, user_id=c279954e82e14184b669a3bcedf77ca9, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:52.359506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f351f61-8ff5-4a6e-af41-e9c911b94a1f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:52.373962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e136a2576c3475481e580f8debfc295', 'request_id': 'req-4065a0e9-ea5b-428a-9ff0-334d26e4881b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.388197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e136a2576c3475481e580f8debfc295', 'request_id': 'req-4065a0e9-ea5b-428a-9ff0-334d26e4881b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.395475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 831/1674] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:52 2022] PUT /identity/v3/projects/ddd7e3061b8d42d6b547bfc99a119f0e/users/c279954e82e14184b669a3bcedf77ca9/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:52.396988 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534766c-3489-4302-a946-8dbc02fd44f6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.397274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534766c-3489-4302-a946-8dbc02fd44f6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.397548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534766c-3489-4302-a946-8dbc02fd44f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.409010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e136a2576c3475481e580f8debfc295', 'request_id': 'req-4065a0e9-ea5b-428a-9ff0-334d26e4881b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.432260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] The token cache is being invalidate because project 8e136a2576c3475481e580f8debfc295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:29:52.432653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.433935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4065a0e9-ea5b-428a-9ff0-334d26e4881b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:52.445457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 844/1675] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:29:52 2022] DELETE /identity/v3/projects/8e136a2576c3475481e580f8debfc295 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:52.446197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b534766c-3489-4302-a946-8dbc02fd44f6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:52.446832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f905f5e-c639-451d-9f8f-e478b8b86cb8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.447020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b534766c-3489-4302-a946-8dbc02fd44f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:52.447371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f905f5e-c639-451d-9f8f-e478b8b86cb8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.447655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f905f5e-c639-451d-9f8f-e478b8b86cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.514623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-9f905f5e-c639-451d-9f8f-e478b8b86cb8 None None] MFA Rules not processed for user `c279954e82e14184b669a3bcedf77ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:52.515873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9f905f5e-c639-451d-9f8f-e478b8b86cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:52.550694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:52.573587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 832/1676] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:52.575028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac23139-d78d-401b-9ef0-9a477bfafa51 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.575311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac23139-d78d-401b-9ef0-9a477bfafa51 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.575582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac23139-d78d-401b-9ef0-9a477bfafa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.624367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3ac23139-d78d-401b-9ef0-9a477bfafa51 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:52.625636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3ac23139-d78d-401b-9ef0-9a477bfafa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:52.646517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 833/1677] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:52.647848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9f3b-86df-4d8f-9502-0725cc02a2b3 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.648077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9f3b-86df-4d8f-9502-0725cc02a2b3 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.648350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe9f3b-86df-4d8f-9502-0725cc02a2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.670779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 845/1678] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:52.672207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b12ef-7c02-4012-89cc-6c9c12d58d9c None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.672529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b12ef-7c02-4012-89cc-6c9c12d58d9c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.672958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b12ef-7c02-4012-89cc-6c9c12d58d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.697658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-5efe9f3b-86df-4d8f-9502-0725cc02a2b3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:52.698896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5efe9f3b-86df-4d8f-9502-0725cc02a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:52.716858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-638b12ef-7c02-4012-89cc-6c9c12d58d9c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:52.718116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-638b12ef-7c02-4012-89cc-6c9c12d58d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:52.729379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 834/1679] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:52.730284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe9f3b-86df-4d8f-9502-0725cc02a2b3 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:52.736912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 846/1680] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:52.738655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5efe9f3b-86df-4d8f-9502-0725cc02a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:52.741154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b12ef-7c02-4012-89cc-6c9c12d58d9c None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:52.761955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe9f3b-86df-4d8f-9502-0725cc02a2b3 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:52.763043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b12ef-7c02-4012-89cc-6c9c12d58d9c None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:52.775681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff77e3-a633-4656-8274-1ade8eeddc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:52.776319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff77e3-a633-4656-8274-1ade8eeddc75 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.776636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff77e3-a633-4656-8274-1ade8eeddc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.776758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d05ac-c233-4634-8ee1-320b8ac5f3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:52.777184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff77e3-a633-4656-8274-1ade8eeddc75 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.777543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d05ac-c233-4634-8ee1-320b8ac5f3eb admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.777655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff77e3-a633-4656-8274-1ade8eeddc75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:52.777995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d05ac-c233-4634-8ee1-320b8ac5f3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.778405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff77e3-a633-4656-8274-1ade8eeddc75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:52.779147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d05ac-c233-4634-8ee1-320b8ac5f3eb admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.779798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d05ac-c233-4634-8ee1-320b8ac5f3eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:52.781748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d05ac-c233-4634-8ee1-320b8ac5f3eb admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:52.782358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff77e3-a633-4656-8274-1ade8eeddc75 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:52.796332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:52.810184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 847/1681] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:52.811056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff77e3-a633-4656-8274-1ade8eeddc75 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:52.837324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff77e3-a633-4656-8274-1ade8eeddc75 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:52.850078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 835/1682] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:52 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:52.850985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d05ac-c233-4634-8ee1-320b8ac5f3eb admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:52.853283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2188c-3112-4b6e-aa1f-32df7b56294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:52.853903 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2188c-3112-4b6e-aa1f-32df7b56294f admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:52.854184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2188c-3112-4b6e-aa1f-32df7b56294f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:52.854474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2188c-3112-4b6e-aa1f-32df7b56294f admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:52.855095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2188c-3112-4b6e-aa1f-32df7b56294f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:52.855388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2188c-3112-4b6e-aa1f-32df7b56294f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:52.857094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2188c-3112-4b6e-aa1f-32df7b56294f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:52.896019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 848/1683] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:52.897024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2188c-3112-4b6e-aa1f-32df7b56294f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:52.919679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2188c-3112-4b6e-aa1f-32df7b56294f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:53.031673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d05ac-c233-4634-8ee1-320b8ac5f3eb admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:53.045542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144093a-99ed-4a83-b792-63b9fbbd27ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:53.046178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144093a-99ed-4a83-b792-63b9fbbd27ab admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:53.046456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144093a-99ed-4a83-b792-63b9fbbd27ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:53.046718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144093a-99ed-4a83-b792-63b9fbbd27ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:53.047358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144093a-99ed-4a83-b792-63b9fbbd27ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:53.048273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a58f4e-2655-496c-8cee-29666cc03b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:53.049153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a58f4e-2655-496c-8cee-29666cc03b5c admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:53.052794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a58f4e-2655-496c-8cee-29666cc03b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:53.053436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a58f4e-2655-496c-8cee-29666cc03b5c admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:53.054342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a58f4e-2655-496c-8cee-29666cc03b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:53.056441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a58f4e-2655-496c-8cee-29666cc03b5c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:53.057743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144093a-99ed-4a83-b792-63b9fbbd27ab admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:53.116100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 849/1684] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:52 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:53.117016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a58f4e-2655-496c-8cee-29666cc03b5c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:53.143873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 836/1685] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:52 2022] POST /identity/v3/projects => generated 367 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:53.148177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144093a-99ed-4a83-b792-63b9fbbd27ab admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:53.158670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a58f4e-2655-496c-8cee-29666cc03b5c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:53.314366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9eb1d-aa16-4bb8-be57-c468a22bfda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:53.315639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9eb1d-aa16-4bb8-be57-c468a22bfda0 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:53.316075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9eb1d-aa16-4bb8-be57-c468a22bfda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:53.316948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9eb1d-aa16-4bb8-be57-c468a22bfda0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:53.318275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9eb1d-aa16-4bb8-be57-c468a22bfda0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:53.321559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9eb1d-aa16-4bb8-be57-c468a22bfda0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:53.349715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144093a-99ed-4a83-b792-63b9fbbd27ab admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:53.366442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6963ff5-8d7f-4e0b-9d1d-82ab1af847d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:53.368274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6963ff5-8d7f-4e0b-9d1d-82ab1af847d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:53.368944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6963ff5-8d7f-4e0b-9d1d-82ab1af847d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:53.369462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6963ff5-8d7f-4e0b-9d1d-82ab1af847d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:53.371963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6963ff5-8d7f-4e0b-9d1d-82ab1af847d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:53.374658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6963ff5-8d7f-4e0b-9d1d-82ab1af847d7 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:53.416550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 850/1686] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:53 2022] POST /identity/v3/projects => generated 387 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:53.417565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9eb1d-aa16-4bb8-be57-c468a22bfda0 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:53.425536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 837/1687] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:53.433547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6963ff5-8d7f-4e0b-9d1d-82ab1af847d7 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:53.471204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6963ff5-8d7f-4e0b-9d1d-82ab1af847d7 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:53.567584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9eb1d-aa16-4bb8-be57-c468a22bfda0 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:53.577952 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea886dd9-d574-40a7-a5f3-f1cd95b6fabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:53.578600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea886dd9-d574-40a7-a5f3-f1cd95b6fabf admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:53.578880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea886dd9-d574-40a7-a5f3-f1cd95b6fabf admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:53.579683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea886dd9-d574-40a7-a5f3-f1cd95b6fabf admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:53.584359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea886dd9-d574-40a7-a5f3-f1cd95b6fabf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:53.589479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea886dd9-d574-40a7-a5f3-f1cd95b6fabf admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:53.614419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:53.642025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 851/1688] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:53 2022] POST /identity/v3/users => generated 284 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:53.647334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea886dd9-d574-40a7-a5f3-f1cd95b6fabf admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:53.655890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:53.682693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea886dd9-d574-40a7-a5f3-f1cd95b6fabf admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:53.695133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bede9c-ee73-4b96-ade4-22ffff8e63d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:53.696643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bede9c-ee73-4b96-ade4-22ffff8e63d7 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:53.696938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bede9c-ee73-4b96-ade4-22ffff8e63d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:53.697216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bede9c-ee73-4b96-ade4-22ffff8e63d7 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:53.699386 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bede9c-ee73-4b96-ade4-22ffff8e63d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:53.699894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0fee6-1404-4fca-a9ef-d65da1a85ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:53.700606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fee6-1404-4fca-a9ef-d65da1a85ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:53.700892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fee6-1404-4fca-a9ef-d65da1a85ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:53.701172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fee6-1404-4fca-a9ef-d65da1a85ddb admin admin] PATH_INFO: `/v3/projects/9403064a27244cc0bf203d86eac7aa28/users/32f64119cd1942f9a95d82496ca5b9b1/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:53.701280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bede9c-ee73-4b96-ade4-22ffff8e63d7 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:53.741101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0fee6-1404-4fca-a9ef-d65da1a85ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9403064a27244cc0bf203d86eac7aa28, user_id=32f64119cd1942f9a95d82496ca5b9b1, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:53.747401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0fee6-1404-4fca-a9ef-d65da1a85ddb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:53.753313 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 838/1689] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:53 2022] POST /identity/v3/users => generated 294 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:53.759678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bede9c-ee73-4b96-ade4-22ffff8e63d7 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:53.776862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 852/1690] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:53 2022] PUT /identity/v3/projects/9403064a27244cc0bf203d86eac7aa28/users/32f64119cd1942f9a95d82496ca5b9b1/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:53.781613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0fee6-1404-4fca-a9ef-d65da1a85ddb admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:53.786735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bede9c-ee73-4b96-ade4-22ffff8e63d7 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:53.945188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c96d9c-79ae-4a34-a8bd-2fb455f8e432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:53.945820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c96d9c-79ae-4a34-a8bd-2fb455f8e432 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:53.946100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c96d9c-79ae-4a34-a8bd-2fb455f8e432 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:53.946370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c96d9c-79ae-4a34-a8bd-2fb455f8e432 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:53.947026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c96d9c-79ae-4a34-a8bd-2fb455f8e432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:53.958104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c96d9c-79ae-4a34-a8bd-2fb455f8e432 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:53.968307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0fee6-1404-4fca-a9ef-d65da1a85ddb admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:53.978058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe6373-b60e-4759-befd-5138ebc722f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:53.979102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe6373-b60e-4759-befd-5138ebc722f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:53.979644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe6373-b60e-4759-befd-5138ebc722f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:53.980139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe6373-b60e-4759-befd-5138ebc722f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:53.981103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe6373-b60e-4759-befd-5138ebc722f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:53.984144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe6373-b60e-4759-befd-5138ebc722f4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:53.997500 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 839/1691] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:53.999722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c96d9c-79ae-4a34-a8bd-2fb455f8e432 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:54.014517 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 853/1692] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:54.016187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe6373-b60e-4759-befd-5138ebc722f4 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:54.030582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c96d9c-79ae-4a34-a8bd-2fb455f8e432 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:54.034727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe6373-b60e-4759-befd-5138ebc722f4 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:54.039017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de41ea5-1641-4e93-bd06-e64e612722ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:54.039786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de41ea5-1641-4e93-bd06-e64e612722ca admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.040068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de41ea5-1641-4e93-bd06-e64e612722ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.040340 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de41ea5-1641-4e93-bd06-e64e612722ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.041890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de41ea5-1641-4e93-bd06-e64e612722ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:54.043606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113592c-b446-4f7b-9b87-de9d82f82ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:54.043831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de41ea5-1641-4e93-bd06-e64e612722ca admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:54.044689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113592c-b446-4f7b-9b87-de9d82f82ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.045330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113592c-b446-4f7b-9b87-de9d82f82ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.045882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113592c-b446-4f7b-9b87-de9d82f82ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.046871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113592c-b446-4f7b-9b87-de9d82f82ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:54.050778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113592c-b446-4f7b-9b87-de9d82f82ef5 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:54.058260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 840/1693] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:54.058260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de41ea5-1641-4e93-bd06-e64e612722ca admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:54.069404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de41ea5-1641-4e93-bd06-e64e612722ca admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:54.074579 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62fab3-0c95-4428-b27a-0d66afa161d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:54.075304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62fab3-0c95-4428-b27a-0d66afa161d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.075524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 854/1694] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:54.075993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62fab3-0c95-4428-b27a-0d66afa161d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.076281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62fab3-0c95-4428-b27a-0d66afa161d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.077887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113592c-b446-4f7b-9b87-de9d82f82ef5 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:54.078369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62fab3-0c95-4428-b27a-0d66afa161d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:54.081997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62fab3-0c95-4428-b27a-0d66afa161d3 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:54.093111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113592c-b446-4f7b-9b87-de9d82f82ef5 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:54.099709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 841/1695] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:54.099914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62fab3-0c95-4428-b27a-0d66afa161d3 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:54.109179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d92dbd-cf44-40e1-9a57-68ddaec405d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:54.111607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d92dbd-cf44-40e1-9a57-68ddaec405d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.112832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d92dbd-cf44-40e1-9a57-68ddaec405d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.113320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d92dbd-cf44-40e1-9a57-68ddaec405d3 admin admin] PATH_INFO: `/v3/projects/9403064a27244cc0bf203d86eac7aa28/users/32f64119cd1942f9a95d82496ca5b9b1/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.116927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62fab3-0c95-4428-b27a-0d66afa161d3 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:54.118456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d92dbd-cf44-40e1-9a57-68ddaec405d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9403064a27244cc0bf203d86eac7aa28, user_id=32f64119cd1942f9a95d82496ca5b9b1, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:54.125223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30570-1abd-4115-8179-727c64fe9016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:54.127189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d92dbd-cf44-40e1-9a57-68ddaec405d3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:54.127789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30570-1abd-4115-8179-727c64fe9016 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.128474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30570-1abd-4115-8179-727c64fe9016 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.129252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30570-1abd-4115-8179-727c64fe9016 admin admin] PATH_INFO: `/v3/projects/64544810ae6542b798a4253a3e963067/users/44b6b4c2bb804a8e9c1f049fed14bde8/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.155495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 855/1696] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:54 2022] PUT /identity/v3/projects/9403064a27244cc0bf203d86eac7aa28/users/32f64119cd1942f9a95d82496ca5b9b1/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:54.157582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d92dbd-cf44-40e1-9a57-68ddaec405d3 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:54.161931 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30570-1abd-4115-8179-727c64fe9016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64544810ae6542b798a4253a3e963067, user_id=44b6b4c2bb804a8e9c1f049fed14bde8, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:54.168441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a30570-1abd-4115-8179-727c64fe9016 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:54.198819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 842/1697] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:54 2022] PUT /identity/v3/projects/64544810ae6542b798a4253a3e963067/users/44b6b4c2bb804a8e9c1f049fed14bde8/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:54.200558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c9388-7187-49c6-a0ab-fa052c99c408 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.200979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c9388-7187-49c6-a0ab-fa052c99c408 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.203311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c9388-7187-49c6-a0ab-fa052c99c408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.274034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-837c9388-7187-49c6-a0ab-fa052c99c408 None None] MFA Rules not processed for user `32f64119cd1942f9a95d82496ca5b9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:54.275463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-837c9388-7187-49c6-a0ab-fa052c99c408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:54.289427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d92dbd-cf44-40e1-9a57-68ddaec405d3 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:54.398588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:54.437910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc50ad3-3dad-4d4b-8d70-b08fc33591d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:54.439191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc50ad3-3dad-4d4b-8d70-b08fc33591d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.439831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc50ad3-3dad-4d4b-8d70-b08fc33591d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.440067 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc50ad3-3dad-4d4b-8d70-b08fc33591d5 admin admin] PATH_INFO: `/v3/projects/6811b3f8068c426787da9ef1c937f5e8/users/d0313b82cda94f30af3b119d7752c2d8/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.475006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc50ad3-3dad-4d4b-8d70-b08fc33591d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6811b3f8068c426787da9ef1c937f5e8, user_id=d0313b82cda94f30af3b119d7752c2d8, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:54.476177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 843/1698] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:54.477620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80500-4cbb-4000-944b-73befdb40fd2 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.479859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80500-4cbb-4000-944b-73befdb40fd2 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.480189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80500-4cbb-4000-944b-73befdb40fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.481027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc50ad3-3dad-4d4b-8d70-b08fc33591d5 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:54.515656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 856/1699] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:54 2022] PUT /identity/v3/projects/6811b3f8068c426787da9ef1c937f5e8/users/d0313b82cda94f30af3b119d7752c2d8/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:54.517837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc50ad3-3dad-4d4b-8d70-b08fc33591d5 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:54.526110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-abc50ad3-3dad-4d4b-8d70-b08fc33591d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:54.541814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:54.566936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-64f80500-4cbb-4000-944b-73befdb40fd2 None None] MFA Rules not processed for user `44b6b4c2bb804a8e9c1f049fed14bde8`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:54.568584 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-64f80500-4cbb-4000-944b-73befdb40fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:54.646801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc50ad3-3dad-4d4b-8d70-b08fc33591d5 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:54.659693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0459e99-507d-455c-83b9-d98d8937d785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:54.660327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0459e99-507d-455c-83b9-d98d8937d785 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.660631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0459e99-507d-455c-83b9-d98d8937d785 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.660907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0459e99-507d-455c-83b9-d98d8937d785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.705272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 844/1700] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:54.705272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6dee0-65f3-48be-a4c3-fc925cc25331 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.705272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6dee0-65f3-48be-a4c3-fc925cc25331 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.705272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6dee0-65f3-48be-a4c3-fc925cc25331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.765943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-9bf6dee0-65f3-48be-a4c3-fc925cc25331 None None] MFA Rules not processed for user `d0313b82cda94f30af3b119d7752c2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:54.773256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9bf6dee0-65f3-48be-a4c3-fc925cc25331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:54.793005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0459e99-507d-455c-83b9-d98d8937d785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:54.794471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0459e99-507d-455c-83b9-d98d8937d785 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:54.814623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 857/1701] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:54.816469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50715ec2-ab59-4ee9-8fd9-6a260d2291b7 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.819522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50715ec2-ab59-4ee9-8fd9-6a260d2291b7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.819998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50715ec2-ab59-4ee9-8fd9-6a260d2291b7 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.823450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 858/1702] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:29:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:29:54.824138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50715ec2-ab59-4ee9-8fd9-6a260d2291b7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:54.831238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-50715ec2-ab59-4ee9-8fd9-6a260d2291b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:54.865541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50715ec2-ab59-4ee9-8fd9-6a260d2291b7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:54.870558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c49bf-6890-41bb-ac60-f2c8e467209a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:54.871408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c49bf-6890-41bb-ac60-f2c8e467209a service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:54.871875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c49bf-6890-41bb-ac60-f2c8e467209a service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:54.872605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c49bf-6890-41bb-ac60-f2c8e467209a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:54.894170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c49bf-6890-41bb-ac60-f2c8e467209a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:54.895902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c49bf-6890-41bb-ac60-f2c8e467209a service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:54.926213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 859/1703] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:54.951369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 845/1704] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:54.956637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c49bf-6890-41bb-ac60-f2c8e467209a service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:55.113514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cd0fa-10d8-46ce-9de8-43d761f96d97 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:55.113927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cd0fa-10d8-46ce-9de8-43d761f96d97 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:55.114109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cd0fa-10d8-46ce-9de8-43d761f96d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:55.138092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c49bf-6890-41bb-ac60-f2c8e467209a service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:55.152867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-5e0cd0fa-10d8-46ce-9de8-43d761f96d97 None None] MFA Rules not processed for user `e30caaea6d104a53a8654442c2383b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:55.153072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361787f-82bf-4f87-b920-b19361a346fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:55.153390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361787f-82bf-4f87-b920-b19361a346fa admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:55.153670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361787f-82bf-4f87-b920-b19361a346fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:55.153947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361787f-82bf-4f87-b920-b19361a346fa admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:55.154675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361787f-82bf-4f87-b920-b19361a346fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:55.155279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5e0cd0fa-10d8-46ce-9de8-43d761f96d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:55.156539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361787f-82bf-4f87-b920-b19361a346fa admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:55.212604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:55.259570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 860/1705] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:54 2022] POST /identity/v3/projects => generated 385 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:55.260103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361787f-82bf-4f87-b920-b19361a346fa admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:55.298513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 846/1706] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:29:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:29:55.449404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cd0fa-10d8-46ce-9de8-43d761f96d97 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:55.452285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5e0cd0fa-10d8-46ce-9de8-43d761f96d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:55.502322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361787f-82bf-4f87-b920-b19361a346fa admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:55.510639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cd0fa-10d8-46ce-9de8-43d761f96d97 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:55.560258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737fe7e-32b0-45b0-8765-000104ec7a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:55.564658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737fe7e-32b0-45b0-8765-000104ec7a54 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:55.564658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737fe7e-32b0-45b0-8765-000104ec7a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:55.566656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737fe7e-32b0-45b0-8765-000104ec7a54 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:55.569585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737fe7e-32b0-45b0-8765-000104ec7a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:55.589024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737fe7e-32b0-45b0-8765-000104ec7a54 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:55.638605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:55.638779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 861/1707] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:29:55 2022] POST /identity/v3/users => generated 299 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:29:55.642487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737fe7e-32b0-45b0-8765-000104ec7a54 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:55.678930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737fe7e-32b0-45b0-8765-000104ec7a54 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:55.696342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b05e9-55ab-4ea6-b57d-762d9eedee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:55.697363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b05e9-55ab-4ea6-b57d-762d9eedee93 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:55.697677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b05e9-55ab-4ea6-b57d-762d9eedee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:55.700812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b05e9-55ab-4ea6-b57d-762d9eedee93 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:55.702588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b05e9-55ab-4ea6-b57d-762d9eedee93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:55.708330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b05e9-55ab-4ea6-b57d-762d9eedee93 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:55.748242 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 862/1708] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:55.749563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b05e9-55ab-4ea6-b57d-762d9eedee93 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:55.770159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb95c9-0534-4d16-8e51-70a7ecb64ae7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:55.770675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb95c9-0534-4d16-8e51-70a7ecb64ae7 service placement] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:55.770952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb95c9-0534-4d16-8e51-70a7ecb64ae7 service placement] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:55.776273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb95c9-0534-4d16-8e51-70a7ecb64ae7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:55.812047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb95c9-0534-4d16-8e51-70a7ecb64ae7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:55.834836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb95c9-0534-4d16-8e51-70a7ecb64ae7 service placement] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:55.897310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 847/1709] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:55.897986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb95c9-0534-4d16-8e51-70a7ecb64ae7 service placement] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:55.912087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb95c9-0534-4d16-8e51-70a7ecb64ae7 service placement] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:55.926964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c4fcd-1fdb-4419-bb81-e8926ce03f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:55.927730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c4fcd-1fdb-4419-bb81-e8926ce03f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:55.928017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c4fcd-1fdb-4419-bb81-e8926ce03f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:55.928285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c4fcd-1fdb-4419-bb81-e8926ce03f19 admin admin] PATH_INFO: `/v3/projects/755be6cfd2f44aabbd5cc4298ddb977e/users/bbc19ad586b54fc6905b44d9fd2fd7d4/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:55.967632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c4fcd-1fdb-4419-bb81-e8926ce03f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755be6cfd2f44aabbd5cc4298ddb977e, user_id=bbc19ad586b54fc6905b44d9fd2fd7d4, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:55.971775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c4fcd-1fdb-4419-bb81-e8926ce03f19 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:55.982257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b05e9-55ab-4ea6-b57d-762d9eedee93 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:55.992385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 848/1710] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:55 2022] PUT /identity/v3/projects/755be6cfd2f44aabbd5cc4298ddb977e/users/bbc19ad586b54fc6905b44d9fd2fd7d4/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:55.996665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c4fcd-1fdb-4419-bb81-e8926ce03f19 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:56.091177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c4fcd-1fdb-4419-bb81-e8926ce03f19 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:56.097856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4071f-3e0d-42bb-9ed8-2ebc52290042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:56.098864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4071f-3e0d-42bb-9ed8-2ebc52290042 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:56.099243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4071f-3e0d-42bb-9ed8-2ebc52290042 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:56.099622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4071f-3e0d-42bb-9ed8-2ebc52290042 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:56.100754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4071f-3e0d-42bb-9ed8-2ebc52290042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:56.103186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4071f-3e0d-42bb-9ed8-2ebc52290042 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:56.126450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 849/1711] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:29:56.127906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4071f-3e0d-42bb-9ed8-2ebc52290042 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:56.157775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4071f-3e0d-42bb-9ed8-2ebc52290042 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:56.163672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d81f42-5944-4065-8359-9773b96ed25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:56.163947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bad6d-116b-4ad2-817a-3f731c5267cf tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f64119cd1942f9a95d82496ca5b9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9403064a27244cc0bf203d86eac7aa28', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:56.164298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d81f42-5944-4065-8359-9773b96ed25b admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:56.164581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bad6d-116b-4ad2-817a-3f731c5267cf tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:56.164771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d81f42-5944-4065-8359-9773b96ed25b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:56.164867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bad6d-116b-4ad2-817a-3f731c5267cf tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:56.165055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d81f42-5944-4065-8359-9773b96ed25b admin admin] PATH_INFO: `/v3/projects/755be6cfd2f44aabbd5cc4298ddb977e/users/bbc19ad586b54fc6905b44d9fd2fd7d4/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:56.165142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bad6d-116b-4ad2-817a-3f731c5267cf tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:56.165909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bad6d-116b-4ad2-817a-3f731c5267cf tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:56.166211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bad6d-116b-4ad2-817a-3f731c5267cf tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:56.167258 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d81f42-5944-4065-8359-9773b96ed25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=755be6cfd2f44aabbd5cc4298ddb977e, user_id=bbc19ad586b54fc6905b44d9fd2fd7d4, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:56.167816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bad6d-116b-4ad2-817a-3f731c5267cf tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:56.170336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d81f42-5944-4065-8359-9773b96ed25b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:56.203285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 863/1712] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:56.216562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bad6d-116b-4ad2-817a-3f731c5267cf tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:56.216941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 850/1713] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:29:56 2022] PUT /identity/v3/projects/755be6cfd2f44aabbd5cc4298ddb977e/users/bbc19ad586b54fc6905b44d9fd2fd7d4/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:56.221294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48d184-8b5f-4259-a036-191817d2b712 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:56.221661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48d184-8b5f-4259-a036-191817d2b712 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:56.221978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48d184-8b5f-4259-a036-191817d2b712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:56.233776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bad6d-116b-4ad2-817a-3f731c5267cf tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:56.245304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:56.265727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ed48d184-8b5f-4259-a036-191817d2b712 None None] MFA Rules not processed for user `bbc19ad586b54fc6905b44d9fd2fd7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:56.267737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ed48d184-8b5f-4259-a036-191817d2b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:56.361987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e86ee-16c4-4590-b2db-e2230c4b13d1 tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f64119cd1942f9a95d82496ca5b9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9403064a27244cc0bf203d86eac7aa28', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:56.363064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e86ee-16c4-4590-b2db-e2230c4b13d1 tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:56.363494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e86ee-16c4-4590-b2db-e2230c4b13d1 tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:56.363920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e86ee-16c4-4590-b2db-e2230c4b13d1 tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:56.370069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e86ee-16c4-4590-b2db-e2230c4b13d1 tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:56.373368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e86ee-16c4-4590-b2db-e2230c4b13d1 tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:56.379067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:56.387756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 851/1714] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:29:56.404234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48d184-8b5f-4259-a036-191817d2b712 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:56.439244 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 864/1715] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:29:56 2022] POST /identity/v3/projects => generated 355 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:29:56.440321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e86ee-16c4-4590-b2db-e2230c4b13d1 tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:56.533974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48d184-8b5f-4259-a036-191817d2b712 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:56.602777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e86ee-16c4-4590-b2db-e2230c4b13d1 tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:56.608294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3239fa-30ca-4440-9d75-b223777b0dfe tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f64119cd1942f9a95d82496ca5b9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9403064a27244cc0bf203d86eac7aa28', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:56.609061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3239fa-30ca-4440-9d75-b223777b0dfe tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:56.609293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3239fa-30ca-4440-9d75-b223777b0dfe tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:56.609567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3239fa-30ca-4440-9d75-b223777b0dfe tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:56.610257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3239fa-30ca-4440-9d75-b223777b0dfe tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:29:56.610560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3239fa-30ca-4440-9d75-b223777b0dfe tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:56.612190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3239fa-30ca-4440-9d75-b223777b0dfe tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:56.627557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 865/1716] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:29:56.679097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ca7f5-943d-42a7-a3a3-82309f3d0f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:56.681247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ca7f5-943d-42a7-a3a3-82309f3d0f1e service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:56.681638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ca7f5-943d-42a7-a3a3-82309f3d0f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:56.686747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ca7f5-943d-42a7-a3a3-82309f3d0f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:56.778048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3239fa-30ca-4440-9d75-b223777b0dfe tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:56.791826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3239fa-30ca-4440-9d75-b223777b0dfe tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:56.812785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '32f64119cd1942f9a95d82496ca5b9b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9403064a27244cc0bf203d86eac7aa28', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:56.817495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:56.817831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:56.818638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] PATH_INFO: `/v3/projects/20ca1a6b2af04563bf0df244b368b6c9` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:56.828037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ca7f5-943d-42a7-a3a3-82309f3d0f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:56.829423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Authorizing `identity:delete_project(project_id=20ca1a6b2af04563bf0df244b368b6c9)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:56.830174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ca7f5-943d-42a7-a3a3-82309f3d0f1e service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:56.841161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:56.856808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 852/1717] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:56.991113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20ca1a6b2af04563bf0df244b368b6c9', 'request_id': 'req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.021719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20ca1a6b2af04563bf0df244b368b6c9', 'request_id': 'req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.049660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20ca1a6b2af04563bf0df244b368b6c9', 'request_id': 'req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.056862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ca7f5-943d-42a7-a3a3-82309f3d0f1e service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:57.091101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] The token cache is being invalidate because project 20ca1a6b2af04563bf0df244b368b6c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:29:57.091391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.092065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f76d8bcc-2d00-43ce-afb7-a1719e2ad8ff tempest-VolumeQuotaClassesTest-1713342035 tempest-VolumeQuotaClassesTest-1713342035-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.093987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 866/1718] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:29:56 2022] DELETE /identity/v3/projects/20ca1a6b2af04563bf0df244b368b6c9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:57.182185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ca7f5-943d-42a7-a3a3-82309f3d0f1e service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:57.193600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:57.330434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:57.331781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:57.333840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:57.334879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] PATH_INFO: `/v3/users/d0313b82cda94f30af3b119d7752c2d8` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:57.338427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0313b82cda94f30af3b119d7752c2d8)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:57.345313 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:57.464115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0313b82cda94f30af3b119d7752c2d8'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.465320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0313b82cda94f30af3b119d7752c2d8'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.496267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0313b82cda94f30af3b119d7752c2d8'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.514611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0313b82cda94f30af3b119d7752c2d8'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.538822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0313b82cda94f30af3b119d7752c2d8'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.568426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 853/1719] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:29:57 2022] DELETE /identity/v3/users/d0313b82cda94f30af3b119d7752c2d8 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:29:57.575529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51299a-2bf6-4c19-9eae-16fc88a3d84b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:57.576011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51299a-2bf6-4c19-9eae-16fc88a3d84b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:57.576477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51299a-2bf6-4c19-9eae-16fc88a3d84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:57.605869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:57.618646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:57.649356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ff51299a-2bf6-4c19-9eae-16fc88a3d84b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:57.653887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ff51299a-2bf6-4c19-9eae-16fc88a3d84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:57.685838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:57.796220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe398b0-6e1e-43d0-866e-e7c73991cd7b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:57.808061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:57.812596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:57.813720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:57.814256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] PATH_INFO: `/v3/users/c279954e82e14184b669a3bcedf77ca9` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:57.815878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c279954e82e14184b669a3bcedf77ca9)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:57.818515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:57.822026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 867/1720] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:57.872886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51299a-2bf6-4c19-9eae-16fc88a3d84b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:57.874667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ff51299a-2bf6-4c19-9eae-16fc88a3d84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:57.902294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:57.925282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c279954e82e14184b669a3bcedf77ca9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.926575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c279954e82e14184b669a3bcedf77ca9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.945532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c279954e82e14184b669a3bcedf77ca9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.955757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c279954e82e14184b669a3bcedf77ca9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.969215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bc182f7c-c56e-47af-ae9d-63ed79596e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c279954e82e14184b669a3bcedf77ca9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:57.988904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 854/1721] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:29:57 2022] DELETE /identity/v3/users/c279954e82e14184b669a3bcedf77ca9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:57.997473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e77ea1-4168-4e50-b182-6e8d1cb72d4d None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:57.998619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e77ea1-4168-4e50-b182-6e8d1cb72d4d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:58.000027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e77ea1-4168-4e50-b182-6e8d1cb72d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:58.010641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51299a-2bf6-4c19-9eae-16fc88a3d84b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:58.067630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-68e77ea1-4168-4e50-b182-6e8d1cb72d4d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:58.074271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-68e77ea1-4168-4e50-b182-6e8d1cb72d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:58.172151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a4c4a-a6f4-4bce-8eec-f918672c47f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:58.176064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a4c4a-a6f4-4bce-8eec-f918672c47f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:58.176362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a4c4a-a6f4-4bce-8eec-f918672c47f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:58.177314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a4c4a-a6f4-4bce-8eec-f918672c47f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:58.185842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-aa9a4c4a-a6f4-4bce-8eec-f918672c47f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:58.291871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 855/1722] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:58.294556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2fdd4-05f9-4803-86db-e048a001de05 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:58.294902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2fdd4-05f9-4803-86db-e048a001de05 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:58.295185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2fdd4-05f9-4803-86db-e048a001de05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:58.370864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-aff2fdd4-05f9-4803-86db-e048a001de05 None None] MFA Rules not processed for user `a6f08eaef2d149a9b0236d4bcbe03a92`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:58.372684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-aff2fdd4-05f9-4803-86db-e048a001de05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:58.373492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a4c4a-a6f4-4bce-8eec-f918672c47f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:58.378333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a4c4a-a6f4-4bce-8eec-f918672c47f6 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:58.397465 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 868/1723] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:29:58.399046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b598a-cfc8-4ac0-9c4d-bef34347ee92 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:58.400242 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b598a-cfc8-4ac0-9c4d-bef34347ee92 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:58.400862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b598a-cfc8-4ac0-9c4d-bef34347ee92 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:58.402827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 869/1724] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:29:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:29:58.403603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b598a-cfc8-4ac0-9c4d-bef34347ee92 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:58.409341 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-725b598a-cfc8-4ac0-9c4d-bef34347ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:58.430957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b598a-cfc8-4ac0-9c4d-bef34347ee92 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:58.436333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f3fe6-d756-4f82-96a5-3ff8be87ad34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:58.436988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f3fe6-d756-4f82-96a5-3ff8be87ad34 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:58.437272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f3fe6-d756-4f82-96a5-3ff8be87ad34 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:58.437546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f3fe6-d756-4f82-96a5-3ff8be87ad34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:58.458378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f3fe6-d756-4f82-96a5-3ff8be87ad34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:58.460671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f3fe6-d756-4f82-96a5-3ff8be87ad34 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:58.477070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 870/1725] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:29:58.477944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f3fe6-d756-4f82-96a5-3ff8be87ad34 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:58.480029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-139f3fe6-d756-4f82-96a5-3ff8be87ad34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:58.499309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:58.508560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f3fe6-d756-4f82-96a5-3ff8be87ad34 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:58.518588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:58.519567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:58.520535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:58.520991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] PATH_INFO: `/v3/users/f336183f8533454db87e5c1c39e82e57` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:58.522916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f336183f8533454db87e5c1c39e82e57)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:58.526273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:58.529605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 856/1726] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:58.534453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec36234-bc7d-4473-99df-7543601e00df None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:58.535530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec36234-bc7d-4473-99df-7543601e00df None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:58.536170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec36234-bc7d-4473-99df-7543601e00df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:58.582602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-9ec36234-bc7d-4473-99df-7543601e00df None None] MFA Rules not processed for user `e30caaea6d104a53a8654442c2383b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:58.590699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9ec36234-bc7d-4473-99df-7543601e00df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:58.683444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:29:58.702866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f336183f8533454db87e5c1c39e82e57'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:58.704054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f336183f8533454db87e5c1c39e82e57'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:58.719198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f336183f8533454db87e5c1c39e82e57'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:58.734661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f336183f8533454db87e5c1c39e82e57'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:58.735858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f336183f8533454db87e5c1c39e82e57'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:29:58.751681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 857/1727] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:29:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:58.753037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf5114-47d7-4cb4-bbbe-253e21492c26 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:58.755456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf5114-47d7-4cb4-bbbe-253e21492c26 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:58.756915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf5114-47d7-4cb4-bbbe-253e21492c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:58.764022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 871/1728] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:29:58 2022] DELETE /identity/v3/users/f336183f8533454db87e5c1c39e82e57 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:29:58.795662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:58.799773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:58.808477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-43cf5114-47d7-4cb4-bbbe-253e21492c26 None None] MFA Rules not processed for user `a6f08eaef2d149a9b0236d4bcbe03a92`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:29:58.809666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-43cf5114-47d7-4cb4-bbbe-253e21492c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:58.918889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7e6fe-6666-4391-b3f6-cc07baa28dfc admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:58.928961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1b704-6956-43fa-8f1f-f354e9659c9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:58.929712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1b704-6956-43fa-8f1f-f354e9659c9a service placement] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:58.931027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1b704-6956-43fa-8f1f-f354e9659c9a service placement] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:58.931312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1b704-6956-43fa-8f1f-f354e9659c9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:58.951612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1b704-6956-43fa-8f1f-f354e9659c9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:58.956331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1b704-6956-43fa-8f1f-f354e9659c9a service placement] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:58.968764 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 858/1729] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:29:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:29:58.981070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf5114-47d7-4cb4-bbbe-253e21492c26 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:29:58.986369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-43cf5114-47d7-4cb4-bbbe-253e21492c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:58.988541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 872/1730] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:29:59.086218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf5114-47d7-4cb4-bbbe-253e21492c26 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:29:59.097039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c1ea2-50e4-48bc-8ffe-d5f58e8f7c45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:29:59.097902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1ea2-50e4-48bc-8ffe-d5f58e8f7c45 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:29:59.098342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1ea2-50e4-48bc-8ffe-d5f58e8f7c45 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:29:59.098901 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c1ea2-50e4-48bc-8ffe-d5f58e8f7c45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:29:59.101961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4d8c1ea2-50e4-48bc-8ffe-d5f58e8f7c45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:29:59.142669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c1ea2-50e4-48bc-8ffe-d5f58e8f7c45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:29:59.148590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c1ea2-50e4-48bc-8ffe-d5f58e8f7c45 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:29:59.180373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 859/1731] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:29:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:00.186038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1b704-6956-43fa-8f1f-f354e9659c9a service placement] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:00.189878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-74a1b704-6956-43fa-8f1f-f354e9659c9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:00.470720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1b704-6956-43fa-8f1f-f354e9659c9a service placement] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:00.483866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:00.486505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:00.486975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:00.488087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] PATH_INFO: `/v3/projects/6811b3f8068c426787da9ef1c937f5e8` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:00.491457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6811b3f8068c426787da9ef1c937f5e8)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:00.497917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:00.522735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:00.605264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f1b3d-3f6e-41db-b5f7-b7c8eac95579 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:00.605841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f1b3d-3f6e-41db-b5f7-b7c8eac95579 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:00.606170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f1b3d-3f6e-41db-b5f7-b7c8eac95579 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:00.607867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 860/1732] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 1 00:30:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 00:30:00.622649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fe45c-b7ac-4038-94ff-397be462b64b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:00.622934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fe45c-b7ac-4038-94ff-397be462b64b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:00.623467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923fe45c-b7ac-4038-94ff-397be462b64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:00.664360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6811b3f8068c426787da9ef1c937f5e8', 'request_id': 'req-a3942dc5-76a7-4037-bc65-bd79328c2566'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:00.685958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6811b3f8068c426787da9ef1c937f5e8', 'request_id': 'req-a3942dc5-76a7-4037-bc65-bd79328c2566'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:00.717908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6811b3f8068c426787da9ef1c937f5e8', 'request_id': 'req-a3942dc5-76a7-4037-bc65-bd79328c2566'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:00.745365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] The token cache is being invalidate because project 6811b3f8068c426787da9ef1c937f5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:00.745365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:00.746740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:00.749820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 873/1733] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:00 2022] DELETE /identity/v3/projects/6811b3f8068c426787da9ef1c937f5e8 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:00.754881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:00.757285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:00.768612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-923fe45c-b7ac-4038-94ff-397be462b64b None None] MFA Rules not processed for user `cdf97e9e0340441c87316ce03d46cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:00.771884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-923fe45c-b7ac-4038-94ff-397be462b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:00.945175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 861/1734] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:30:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:00.946732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5ec63-d308-494b-a7ec-93f096e70a87 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:00.947622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5ec63-d308-494b-a7ec-93f096e70a87 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:00.948052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5ec63-d308-494b-a7ec-93f096e70a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:00.961223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3942dc5-76a7-4037-bc65-bd79328c2566 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:00.971538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:00.978295 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:00.978959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:00.979235 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:00.979504 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] PATH_INFO: `/v3/users/bbc19ad586b54fc6905b44d9fd2fd7d4` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:00.980701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc19ad586b54fc6905b44d9fd2fd7d4)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:00.982652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:01.033435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-abf5ec63-d308-494b-a7ec-93f096e70a87 None None] MFA Rules not processed for user `5bf265cb4af14e2d8815c42288e2a1f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:01.035693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-abf5ec63-d308-494b-a7ec-93f096e70a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:01.111907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc19ad586b54fc6905b44d9fd2fd7d4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.116940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc19ad586b54fc6905b44d9fd2fd7d4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.141797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc19ad586b54fc6905b44d9fd2fd7d4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.162175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc19ad586b54fc6905b44d9fd2fd7d4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.163360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc19ad586b54fc6905b44d9fd2fd7d4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.205602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 874/1735] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:00 2022] DELETE /identity/v3/users/bbc19ad586b54fc6905b44d9fd2fd7d4 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:01.206652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:01.220286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:01.259538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 862/1736] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:01.260453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5ec63-d308-494b-a7ec-93f096e70a87 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:01.271214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-abf5ec63-d308-494b-a7ec-93f096e70a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:01.276758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:01.309519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5ec63-d308-494b-a7ec-93f096e70a87 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:01.470688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3183ff6-33e6-4961-943f-3d7cd458a61c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:01.490723 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:01.492706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:01.492987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:01.503857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] PATH_INFO: `/v3/projects/ddd7e3061b8d42d6b547bfc99a119f0e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:01.505312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddd7e3061b8d42d6b547bfc99a119f0e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:01.507262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:01.540175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:01.540879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:01.541158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:01.541433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] PATH_INFO: `/v3/projects/b620fa18ad9947a8b810be774b15e7c1` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:01.542689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b620fa18ad9947a8b810be774b15e7c1)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:01.544720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:01.645440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b620fa18ad9947a8b810be774b15e7c1', 'request_id': 'req-9c6a8792-e958-41b0-8171-b8a6258516ad'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.653403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b620fa18ad9947a8b810be774b15e7c1', 'request_id': 'req-9c6a8792-e958-41b0-8171-b8a6258516ad'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.664148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b620fa18ad9947a8b810be774b15e7c1', 'request_id': 'req-9c6a8792-e958-41b0-8171-b8a6258516ad'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.665218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddd7e3061b8d42d6b547bfc99a119f0e', 'request_id': 'req-17130247-8b55-4a0b-aa2c-d474ba17d3d7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.676326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddd7e3061b8d42d6b547bfc99a119f0e', 'request_id': 'req-17130247-8b55-4a0b-aa2c-d474ba17d3d7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.686641 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] The token cache is being invalidate because project b620fa18ad9947a8b810be774b15e7c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:01.687364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.688804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.690631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 863/1737] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:01 2022] DELETE /identity/v3/projects/b620fa18ad9947a8b810be774b15e7c1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:01.691662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:01.693564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:01.702394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddd7e3061b8d42d6b547bfc99a119f0e', 'request_id': 'req-17130247-8b55-4a0b-aa2c-d474ba17d3d7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.722134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] The token cache is being invalidate because project ddd7e3061b8d42d6b547bfc99a119f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:01.722437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.726877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-17130247-8b55-4a0b-aa2c-d474ba17d3d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:01.735125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 875/1738] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:01 2022] DELETE /identity/v3/projects/ddd7e3061b8d42d6b547bfc99a119f0e => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:01.738877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73423a7-3dda-4713-83a7-5689ba00535b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:01.739216 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73423a7-3dda-4713-83a7-5689ba00535b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:01.740032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73423a7-3dda-4713-83a7-5689ba00535b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:01.759107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:01.779213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b73423a7-3dda-4713-83a7-5689ba00535b None None] MFA Rules not processed for user `ecf29db866eb43fa954b0d440577f45f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:01.780648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b73423a7-3dda-4713-83a7-5689ba00535b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:01.809243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a8792-e958-41b0-8171-b8a6258516ad admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:01.871785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 876/1739] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:01.871785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73423a7-3dda-4713-83a7-5689ba00535b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:01.871785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b73423a7-3dda-4713-83a7-5689ba00535b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:01.918662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:01.934339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:01.936014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:01.936141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:01.936365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:01.941743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:01.966047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:01.968003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:01.984315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73423a7-3dda-4713-83a7-5689ba00535b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:01.986540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 864/1740] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:01.987522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:01.988943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:01.991085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194fdb4-1c2a-4f6e-8a5d-33c5ef8a1984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:01.991790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194fdb4-1c2a-4f6e-8a5d-33c5ef8a1984 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:01.992075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194fdb4-1c2a-4f6e-8a5d-33c5ef8a1984 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:01.992365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194fdb4-1c2a-4f6e-8a5d-33c5ef8a1984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:01.994169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3194fdb4-1c2a-4f6e-8a5d-33c5ef8a1984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:02.075260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194fdb4-1c2a-4f6e-8a5d-33c5ef8a1984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:02.077407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194fdb4-1c2a-4f6e-8a5d-33c5ef8a1984 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:02.103924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 877/1741] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:02.106090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318844f-488d-4ae3-9fc9-cbbfeb38cdce None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:02.106319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318844f-488d-4ae3-9fc9-cbbfeb38cdce None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:02.109030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318844f-488d-4ae3-9fc9-cbbfeb38cdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:02.113731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27079-2624-473c-90c1-e92643c836a1 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:02.137270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:02.138271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:02.139385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:02.140152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] PATH_INFO: `/v3/users/ee1a8c58a0ec4a69835f67c2336eb1e6` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:02.145371 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee1a8c58a0ec4a69835f67c2336eb1e6)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:02.158526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:02.185224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b318844f-488d-4ae3-9fc9-cbbfeb38cdce None None] MFA Rules not processed for user `6f67deff06894bffb9ac99680df14b88`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:02.186508 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b318844f-488d-4ae3-9fc9-cbbfeb38cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:02.300378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee1a8c58a0ec4a69835f67c2336eb1e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.301163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee1a8c58a0ec4a69835f67c2336eb1e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.338879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee1a8c58a0ec4a69835f67c2336eb1e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.358036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee1a8c58a0ec4a69835f67c2336eb1e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.377103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee1a8c58a0ec4a69835f67c2336eb1e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.386197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 865/1742] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:01 2022] DELETE /identity/v3/users/ee1a8c58a0ec4a69835f67c2336eb1e6 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:02.390252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:02.392233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:02.399177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:02.419169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 878/1743] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:02.422183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318844f-488d-4ae3-9fc9-cbbfeb38cdce None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:02.426699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b318844f-488d-4ae3-9fc9-cbbfeb38cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:02.474949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318844f-488d-4ae3-9fc9-cbbfeb38cdce None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:02.551258 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68898b-d482-44e8-991c-ddd636ad3d82 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:02.557989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:02.558920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:02.559352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:02.559787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] PATH_INFO: `/v3/users/32f64119cd1942f9a95d82496ca5b9b1` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:02.561345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f64119cd1942f9a95d82496ca5b9b1)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:02.563924 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:02.673287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f64119cd1942f9a95d82496ca5b9b1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.674032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f64119cd1942f9a95d82496ca5b9b1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.679295 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:02.680337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:02.682433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:02.683125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] PATH_INFO: `/v3/projects/755be6cfd2f44aabbd5cc4298ddb977e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:02.685024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=755be6cfd2f44aabbd5cc4298ddb977e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:02.687573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:02.689231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f64119cd1942f9a95d82496ca5b9b1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.702568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:02.709179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f64119cd1942f9a95d82496ca5b9b1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.728324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-95dc55ee-23df-4646-94de-35d6c3b612d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f64119cd1942f9a95d82496ca5b9b1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.750136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 866/1744] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:02 2022] DELETE /identity/v3/users/32f64119cd1942f9a95d82496ca5b9b1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:02.752486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1705d0-cf32-4097-9dbe-2a264e2f90a5 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:02.757844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1705d0-cf32-4097-9dbe-2a264e2f90a5 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:02.758172 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1705d0-cf32-4097-9dbe-2a264e2f90a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:02.843769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-cc1705d0-cf32-4097-9dbe-2a264e2f90a5 None None] MFA Rules not processed for user `6f67deff06894bffb9ac99680df14b88`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:02.846179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-cc1705d0-cf32-4097-9dbe-2a264e2f90a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:02.970095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 867/1745] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:02.970407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755be6cfd2f44aabbd5cc4298ddb977e', 'request_id': 'req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.972625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1705d0-cf32-4097-9dbe-2a264e2f90a5 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:02.976189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-cc1705d0-cf32-4097-9dbe-2a264e2f90a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:02.989460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755be6cfd2f44aabbd5cc4298ddb977e', 'request_id': 'req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:02.998343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755be6cfd2f44aabbd5cc4298ddb977e', 'request_id': 'req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:03.027729 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] The token cache is being invalidate because project 755be6cfd2f44aabbd5cc4298ddb977e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:03.028389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:03.029595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:03.037402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 879/1746] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:02 2022] DELETE /identity/v3/projects/755be6cfd2f44aabbd5cc4298ddb977e => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:03.040996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:03.041510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:03.045463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:03.163471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1705d0-cf32-4097-9dbe-2a264e2f90a5 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:03.196790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0b7e1-7d4b-40a9-a6c1-55281510a8cb admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:03.202844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f15d0-400a-4943-8e4e-2f9ddc9fd21f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:03.203920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f15d0-400a-4943-8e4e-2f9ddc9fd21f service placement] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:03.204608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f15d0-400a-4943-8e4e-2f9ddc9fd21f service placement] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:03.205050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f15d0-400a-4943-8e4e-2f9ddc9fd21f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:03.207094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b96f15d0-400a-4943-8e4e-2f9ddc9fd21f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:03.231380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f15d0-400a-4943-8e4e-2f9ddc9fd21f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:03.236814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f15d0-400a-4943-8e4e-2f9ddc9fd21f service placement] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:03.251922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 880/1747] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:03.256019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c13ca-1722-4d09-aab4-bb902bfd01d7 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:03.256637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c13ca-1722-4d09-aab4-bb902bfd01d7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:03.257105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c13ca-1722-4d09-aab4-bb902bfd01d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:03.282799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83eb28-cded-42b4-a3a9-0aa49c7017fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:03.283510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83eb28-cded-42b4-a3a9-0aa49c7017fc service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:03.283802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83eb28-cded-42b4-a3a9-0aa49c7017fc service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:03.284077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83eb28-cded-42b4-a3a9-0aa49c7017fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:03.286436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6b83eb28-cded-42b4-a3a9-0aa49c7017fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:03.330711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-698c13ca-1722-4d09-aab4-bb902bfd01d7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:03.332905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-698c13ca-1722-4d09-aab4-bb902bfd01d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:03.362109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83eb28-cded-42b4-a3a9-0aa49c7017fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:03.363761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83eb28-cded-42b4-a3a9-0aa49c7017fc service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:03.378878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 868/1748] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:03.546947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 881/1749] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:03.565214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ee79-31d5-4d79-8922-a16105696727 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:03.565637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ee79-31d5-4d79-8922-a16105696727 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:03.565972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257ee79-31d5-4d79-8922-a16105696727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:03.566151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c13ca-1722-4d09-aab4-bb902bfd01d7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:03.567665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-698c13ca-1722-4d09-aab4-bb902bfd01d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:03.589651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c13ca-1722-4d09-aab4-bb902bfd01d7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:03.615105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e8dfc-bd7f-44dd-9b35-c5174d58d8ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:03.615660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e8dfc-bd7f-44dd-9b35-c5174d58d8ff service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:03.616470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e8dfc-bd7f-44dd-9b35-c5174d58d8ff service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:03.616797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e8dfc-bd7f-44dd-9b35-c5174d58d8ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:03.625225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1257ee79-31d5-4d79-8922-a16105696727 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:03.629947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1257ee79-31d5-4d79-8922-a16105696727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:03.633220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ad5e8dfc-bd7f-44dd-9b35-c5174d58d8ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:03.671161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 869/1750] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:03.672626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ee79-31d5-4d79-8922-a16105696727 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:03.674530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1257ee79-31d5-4d79-8922-a16105696727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:03.692877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257ee79-31d5-4d79-8922-a16105696727 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:03.699107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:03.700655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:03.700944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:03.701223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] PATH_INFO: `/v3/projects/25cc8a74338e49e7b7332b34c30c1c43` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:03.703137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=25cc8a74338e49e7b7332b34c30c1c43)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:03.706923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:03.751042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:03.874842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25cc8a74338e49e7b7332b34c30c1c43', 'request_id': 'req-3186ba92-89c3-42c6-b5c7-b2df88db76ed'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:03.880689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:03.914323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25cc8a74338e49e7b7332b34c30c1c43', 'request_id': 'req-3186ba92-89c3-42c6-b5c7-b2df88db76ed'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:03.959203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25cc8a74338e49e7b7332b34c30c1c43', 'request_id': 'req-3186ba92-89c3-42c6-b5c7-b2df88db76ed'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:03.974826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e8dfc-bd7f-44dd-9b35-c5174d58d8ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:03.981743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e8dfc-bd7f-44dd-9b35-c5174d58d8ff service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:04.007366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] The token cache is being invalidate because project 25cc8a74338e49e7b7332b34c30c1c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:04.010057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:04.014517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:04.023136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 870/1751] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:03 2022] DELETE /identity/v3/projects/25cc8a74338e49e7b7332b34c30c1c43 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:04.026361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:04.080395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 882/1752] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:04.082519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:04.088355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5fa1b-b9a3-4a8e-9e95-70e782b6f110 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:04.091412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5fa1b-b9a3-4a8e-9e95-70e782b6f110 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:04.091780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5fa1b-b9a3-4a8e-9e95-70e782b6f110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:04.187912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3de5fa1b-b9a3-4a8e-9e95-70e782b6f110 None None] MFA Rules not processed for user `a6f08eaef2d149a9b0236d4bcbe03a92`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:04.194491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3de5fa1b-b9a3-4a8e-9e95-70e782b6f110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:04.365803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 883/1753] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:04.368000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b38e3-88e7-4255-9f5c-4668da685ecb None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:04.368296 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b38e3-88e7-4255-9f5c-4668da685ecb None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:04.368666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b38e3-88e7-4255-9f5c-4668da685ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:04.410967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186ba92-89c3-42c6-b5c7-b2df88db76ed admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:04.416598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-435b38e3-88e7-4255-9f5c-4668da685ecb None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:04.423370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-435b38e3-88e7-4255-9f5c-4668da685ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:04.433261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7c40e-d616-48c1-bd51-55d23215a4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:04.435463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7c40e-d616-48c1-bd51-55d23215a4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:04.436958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7c40e-d616-48c1-bd51-55d23215a4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:04.437557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7c40e-d616-48c1-bd51-55d23215a4b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:04.439732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7c40e-d616-48c1-bd51-55d23215a4b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:04.440054 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7c40e-d616-48c1-bd51-55d23215a4b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:04.447624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7c40e-d616-48c1-bd51-55d23215a4b2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:04.488468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 884/1754] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:04.501939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58717aa-1991-4293-bada-039a9485d992 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:04.501939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58717aa-1991-4293-bada-039a9485d992 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:04.501939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58717aa-1991-4293-bada-039a9485d992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:04.532238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 871/1755] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 508 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:04.536796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7c40e-d616-48c1-bd51-55d23215a4b2 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:04.577295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7c40e-d616-48c1-bd51-55d23215a4b2 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:04.585074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a58717aa-1991-4293-bada-039a9485d992 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:04.590711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1e1e5-c2b5-4557-8758-50670c09d927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:04.590835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a58717aa-1991-4293-bada-039a9485d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:04.591342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1e1e5-c2b5-4557-8758-50670c09d927 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:04.591629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1e1e5-c2b5-4557-8758-50670c09d927 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:04.592437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1e1e5-c2b5-4557-8758-50670c09d927 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:04.592652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1e1e5-c2b5-4557-8758-50670c09d927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:04.595155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1e1e5-c2b5-4557-8758-50670c09d927 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:04.629281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 885/1756] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:04.638888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58717aa-1991-4293-bada-039a9485d992 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:04.659275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 872/1757] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:04 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:04.663318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1e1e5-c2b5-4557-8758-50670c09d927 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:04.670336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58717aa-1991-4293-bada-039a9485d992 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:04.851164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3a9fd-c915-47ea-8a5d-ba65d621653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:04.851855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3a9fd-c915-47ea-8a5d-ba65d621653a admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:04.852139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3a9fd-c915-47ea-8a5d-ba65d621653a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:04.852410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3a9fd-c915-47ea-8a5d-ba65d621653a admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:04.853655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3a9fd-c915-47ea-8a5d-ba65d621653a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:04.853655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3a9fd-c915-47ea-8a5d-ba65d621653a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:04.860831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3a9fd-c915-47ea-8a5d-ba65d621653a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:04.867083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1e1e5-c2b5-4557-8758-50670c09d927 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:04.878437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f5f04-71fc-4bde-8e29-80304cae19c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:04.879064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f5f04-71fc-4bde-8e29-80304cae19c5 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:04.879341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f5f04-71fc-4bde-8e29-80304cae19c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:04.879617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f5f04-71fc-4bde-8e29-80304cae19c5 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:04.880247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f5f04-71fc-4bde-8e29-80304cae19c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:04.882228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f5f04-71fc-4bde-8e29-80304cae19c5 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:04.907470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 886/1758] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:04.908911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3a9fd-c915-47ea-8a5d-ba65d621653a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:04.915453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 873/1759] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:04 2022] POST /identity/v3/users => generated 292 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:04.919618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f5f04-71fc-4bde-8e29-80304cae19c5 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:04.939698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3a9fd-c915-47ea-8a5d-ba65d621653a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:04.940815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f5f04-71fc-4bde-8e29-80304cae19c5 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:04.952683 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b547c-da35-4444-a72c-4c700ca6cea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:04.953213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b547c-da35-4444-a72c-4c700ca6cea6 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:04.955301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b547c-da35-4444-a72c-4c700ca6cea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:04.955577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b547c-da35-4444-a72c-4c700ca6cea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:04.956857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ab56d-83c3-4712-b8a1-590bc1cfbf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:04.957497 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ab56d-83c3-4712-b8a1-590bc1cfbf4c admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:04.957781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ab56d-83c3-4712-b8a1-590bc1cfbf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:04.957944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b547c-da35-4444-a72c-4c700ca6cea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:04.958050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ab56d-83c3-4712-b8a1-590bc1cfbf4c admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:04.959638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ab56d-83c3-4712-b8a1-590bc1cfbf4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:04.961052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ab56d-83c3-4712-b8a1-590bc1cfbf4c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:04.962057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b547c-da35-4444-a72c-4c700ca6cea6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:04.993459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 874/1760] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:04.997403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b547c-da35-4444-a72c-4c700ca6cea6 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:05.025373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b547c-da35-4444-a72c-4c700ca6cea6 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:05.040343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5452ad4-adca-42bc-918f-8af2d332426a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:05.041041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5452ad4-adca-42bc-918f-8af2d332426a admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:05.041312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5452ad4-adca-42bc-918f-8af2d332426a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:05.041571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5452ad4-adca-42bc-918f-8af2d332426a admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:05.042223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5452ad4-adca-42bc-918f-8af2d332426a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:05.043858 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5452ad4-adca-42bc-918f-8af2d332426a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:05.073935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 887/1761] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:04 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:05.078094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ab56d-83c3-4712-b8a1-590bc1cfbf4c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:05.081046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 875/1762] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:30:05.085268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5452ad4-adca-42bc-918f-8af2d332426a admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:05.111087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5452ad4-adca-42bc-918f-8af2d332426a admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:05.170676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:05.170859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:05.286018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ab56d-83c3-4712-b8a1-590bc1cfbf4c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:05.300543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801102-6fc5-47cc-9c5e-da3e772a4914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:05.302215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801102-6fc5-47cc-9c5e-da3e772a4914 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:05.303867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801102-6fc5-47cc-9c5e-da3e772a4914 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:05.304309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801102-6fc5-47cc-9c5e-da3e772a4914 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:05.306285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8801102-6fc5-47cc-9c5e-da3e772a4914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:05.308459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8801102-6fc5-47cc-9c5e-da3e772a4914 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:05.310434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716a021-1a8e-47a4-a19e-9ec93a3f69ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:05.313243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716a021-1a8e-47a4-a19e-9ec93a3f69ea admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:05.314321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716a021-1a8e-47a4-a19e-9ec93a3f69ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:05.316489 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716a021-1a8e-47a4-a19e-9ec93a3f69ea admin admin] PATH_INFO: `/v3/projects/65376d442da84facae3d0473b04a4b3e/users/2b82b880d82945478ffa8d811341d095/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:05.374420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716a021-1a8e-47a4-a19e-9ec93a3f69ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=65376d442da84facae3d0473b04a4b3e, user_id=2b82b880d82945478ffa8d811341d095, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:05.382427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716a021-1a8e-47a4-a19e-9ec93a3f69ea admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:05.400026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 888/1763] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:05 2022] POST /identity/v3/users => generated 281 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:05.401221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801102-6fc5-47cc-9c5e-da3e772a4914 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:05.407734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f8801102-6fc5-47cc-9c5e-da3e772a4914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:05.438314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 876/1764] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:05 2022] PUT /identity/v3/projects/65376d442da84facae3d0473b04a4b3e/users/2b82b880d82945478ffa8d811341d095/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:05.439315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716a021-1a8e-47a4-a19e-9ec93a3f69ea admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:05.453420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801102-6fc5-47cc-9c5e-da3e772a4914 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:05.639187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716a021-1a8e-47a4-a19e-9ec93a3f69ea admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:05.645160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:05.645824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:05.646102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:05.646376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] PATH_INFO: `/v3/projects/9403064a27244cc0bf203d86eac7aa28` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:05.647974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9403064a27244cc0bf203d86eac7aa28)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:05.649951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:05.656079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de11c3-6fdc-4e50-9958-c6c6011b7b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:05.657770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de11c3-6fdc-4e50-9958-c6c6011b7b0e admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:05.658051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de11c3-6fdc-4e50-9958-c6c6011b7b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:05.658453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de11c3-6fdc-4e50-9958-c6c6011b7b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:05.659122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de11c3-6fdc-4e50-9958-c6c6011b7b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:05.661112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de11c3-6fdc-4e50-9958-c6c6011b7b0e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:05.689629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 877/1765] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:05.691397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee58e84-c741-42ed-acc9-5c47356ead9d None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:05.694632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee58e84-c741-42ed-acc9-5c47356ead9d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:05.695184 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee58e84-c741-42ed-acc9-5c47356ead9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:05.757451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0ee58e84-c741-42ed-acc9-5c47356ead9d None None] MFA Rules not processed for user `2b82b880d82945478ffa8d811341d095`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:05.759179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0ee58e84-c741-42ed-acc9-5c47356ead9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:05.786620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9403064a27244cc0bf203d86eac7aa28', 'request_id': 'req-4bea020f-51ed-4252-9293-8d626237192c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:05.802302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9403064a27244cc0bf203d86eac7aa28', 'request_id': 'req-4bea020f-51ed-4252-9293-8d626237192c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:05.816202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9403064a27244cc0bf203d86eac7aa28', 'request_id': 'req-4bea020f-51ed-4252-9293-8d626237192c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:05.844084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] The token cache is being invalidate because project 9403064a27244cc0bf203d86eac7aa28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:05.844387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:05.845717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:05.852811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 889/1766] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:05 2022] DELETE /identity/v3/projects/9403064a27244cc0bf203d86eac7aa28 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:05.854020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:05.857283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:05.976605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:05.986732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 878/1767] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:05.991496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd5a31-c401-45ee-a521-3abd7e34d16c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:05.992013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd5a31-c401-45ee-a521-3abd7e34d16c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:05.992508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd5a31-c401-45ee-a521-3abd7e34d16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.038075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-a6cd5a31-c401-45ee-a521-3abd7e34d16c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:06.039392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a6cd5a31-c401-45ee-a521-3abd7e34d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:06.042047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea020f-51ed-4252-9293-8d626237192c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:06.069860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 879/1768] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:06.070604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044ce6d-39ec-4241-9259-a6f66eb49484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:06.071250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044ce6d-39ec-4241-9259-a6f66eb49484 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.071531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044ce6d-39ec-4241-9259-a6f66eb49484 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.071810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044ce6d-39ec-4241-9259-a6f66eb49484 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.071922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd5a31-c401-45ee-a521-3abd7e34d16c None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:06.072460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044ce6d-39ec-4241-9259-a6f66eb49484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:06.074219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044ce6d-39ec-4241-9259-a6f66eb49484 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:06.082519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a6cd5a31-c401-45ee-a521-3abd7e34d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:06.140210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd5a31-c401-45ee-a521-3abd7e34d16c None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:06.155479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5c556-6401-4f9b-ae6e-eaf8d96cd8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:06.155479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5c556-6401-4f9b-ae6e-eaf8d96cd8ec admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.155479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5c556-6401-4f9b-ae6e-eaf8d96cd8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.157347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 890/1769] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:06.158091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f5c556-6401-4f9b-ae6e-eaf8d96cd8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.158841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5c556-6401-4f9b-ae6e-eaf8d96cd8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:06.161060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f5c556-6401-4f9b-ae6e-eaf8d96cd8ec admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:06.164227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf214a-fbce-4217-b3c6-41453c14f87e None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.164918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf214a-fbce-4217-b3c6-41453c14f87e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.165466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf214a-fbce-4217-b3c6-41453c14f87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.250133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:06.285261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 880/1770] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:06 2022] POST /identity/v3/projects => generated 381 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:06.286002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-1caf214a-fbce-4217-b3c6-41453c14f87e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:06.289897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1caf214a-fbce-4217-b3c6-41453c14f87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:06.292976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5c556-6401-4f9b-ae6e-eaf8d96cd8ec admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:06.457536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 891/1771] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:06.457536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf214a-fbce-4217-b3c6-41453c14f87e None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:06.479530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf214a-fbce-4217-b3c6-41453c14f87e None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:06.482039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f5c556-6401-4f9b-ae6e-eaf8d96cd8ec admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:06.493156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67517f-e4c8-4a4f-909d-69676bdeae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:06.494603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67517f-e4c8-4a4f-909d-69676bdeae26 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.495737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67517f-e4c8-4a4f-909d-69676bdeae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.496076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67517f-e4c8-4a4f-909d-69676bdeae26 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.497877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67517f-e4c8-4a4f-909d-69676bdeae26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:06.503196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67517f-e4c8-4a4f-909d-69676bdeae26 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:06.504607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6f91f-2bd8-4618-adc7-42d60ce69845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:06.505555 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6f91f-2bd8-4618-adc7-42d60ce69845 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.507088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6f91f-2bd8-4618-adc7-42d60ce69845 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.527010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6f91f-2bd8-4618-adc7-42d60ce69845 admin admin] PATH_INFO: `/v3/projects/d2cb59a42b584d07915d5b1ae4e9e3b6/users/6dcc16a91c094bc7b7636d978e069219/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.539498 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 892/1772] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:06 2022] POST /identity/v3/users => generated 297 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:06.541044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c4132-1691-4bd4-9431-d1af01583bd0 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.541673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c4132-1691-4bd4-9431-d1af01583bd0 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.541954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c4132-1691-4bd4-9431-d1af01583bd0 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.543721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 893/1773] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 1 00:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:30:06.544533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c4132-1691-4bd4-9431-d1af01583bd0 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:06.567708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c4132-1691-4bd4-9431-d1af01583bd0 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:06.569565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6f91f-2bd8-4618-adc7-42d60ce69845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2cb59a42b584d07915d5b1ae4e9e3b6, user_id=6dcc16a91c094bc7b7636d978e069219, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:06.573737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6f91f-2bd8-4618-adc7-42d60ce69845 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:06.590773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e60d56-f8b3-4f54-a518-b10d0ac464b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:06.591420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e60d56-f8b3-4f54-a518-b10d0ac464b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.597387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e60d56-f8b3-4f54-a518-b10d0ac464b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.598347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e60d56-f8b3-4f54-a518-b10d0ac464b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.601032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e60d56-f8b3-4f54-a518-b10d0ac464b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:06.603148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e60d56-f8b3-4f54-a518-b10d0ac464b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:06.606744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 881/1774] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:06 2022] PUT /identity/v3/projects/d2cb59a42b584d07915d5b1ae4e9e3b6/users/6dcc16a91c094bc7b7636d978e069219/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:06.607444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e60d56-f8b3-4f54-a518-b10d0ac464b0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:06.607811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6f91f-2bd8-4618-adc7-42d60ce69845 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:06.663209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 894/1775] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:06.663209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed714f50-e8ae-4828-b662-a1a0b2cc4c4c None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.664044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed714f50-e8ae-4828-b662-a1a0b2cc4c4c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.664044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed714f50-e8ae-4828-b662-a1a0b2cc4c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.719506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ed714f50-e8ae-4828-b662-a1a0b2cc4c4c None None] MFA Rules not processed for user `cdf97e9e0340441c87316ce03d46cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:06.723636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ed714f50-e8ae-4828-b662-a1a0b2cc4c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:06.871188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6f91f-2bd8-4618-adc7-42d60ce69845 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:06.896006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d9eb1-ab3c-48e3-b9b1-f9101eca9180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:06.897689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9eb1-ab3c-48e3-b9b1-f9101eca9180 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.898623 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9eb1-ab3c-48e3-b9b1-f9101eca9180 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.900786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d9eb1-ab3c-48e3-b9b1-f9101eca9180 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:06.901626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d9eb1-ab3c-48e3-b9b1-f9101eca9180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:06.930215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d9eb1-ab3c-48e3-b9b1-f9101eca9180 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:06.960337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 882/1776] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:06.968299 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b0200-4bb1-48f7-ba80-50aabb99c9d8 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:06.968731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b0200-4bb1-48f7-ba80-50aabb99c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:06.969814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b0200-4bb1-48f7-ba80-50aabb99c9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:07.033832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 895/1777] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:30:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:07.036097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed714f50-e8ae-4828-b662-a1a0b2cc4c4c None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:07.044989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-126b0200-4bb1-48f7-ba80-50aabb99c9d8 None None] MFA Rules not processed for user `6dcc16a91c094bc7b7636d978e069219`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:07.046606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-126b0200-4bb1-48f7-ba80-50aabb99c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:07.090743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed714f50-e8ae-4828-b662-a1a0b2cc4c4c None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:07.105259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0909eb1-4b8d-4af4-b3e6-33b0e4241c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:07.105896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0909eb1-4b8d-4af4-b3e6-33b0e4241c95 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:07.106174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0909eb1-4b8d-4af4-b3e6-33b0e4241c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:07.106441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0909eb1-4b8d-4af4-b3e6-33b0e4241c95 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:07.111395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0909eb1-4b8d-4af4-b3e6-33b0e4241c95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:07.115599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0909eb1-4b8d-4af4-b3e6-33b0e4241c95 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:07.181496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 896/1778] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:07 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:07.181496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0909eb1-4b8d-4af4-b3e6-33b0e4241c95 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:07.270484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 883/1779] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:07.271345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b0200-4bb1-48f7-ba80-50aabb99c9d8 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:07.273605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-126b0200-4bb1-48f7-ba80-50aabb99c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:07.290594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b0200-4bb1-48f7-ba80-50aabb99c9d8 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:07.296041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:07.390600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1213d1-e21b-4358-a43d-7bfbc4a39a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:07.391582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1213d1-e21b-4358-a43d-7bfbc4a39a86 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:07.392011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1213d1-e21b-4358-a43d-7bfbc4a39a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:07.392428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1213d1-e21b-4358-a43d-7bfbc4a39a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:07.395452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:07.401796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1213d1-e21b-4358-a43d-7bfbc4a39a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:07.403764 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1213d1-e21b-4358-a43d-7bfbc4a39a86 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:07.416133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 884/1780] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:07.417102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1213d1-e21b-4358-a43d-7bfbc4a39a86 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:07.480298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0909eb1-4b8d-4af4-b3e6-33b0e4241c95 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:07.491993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62439ec-8936-4cff-87f3-3b900aab4be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:07.493989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62439ec-8936-4cff-87f3-3b900aab4be6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:07.495049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62439ec-8936-4cff-87f3-3b900aab4be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:07.495952 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62439ec-8936-4cff-87f3-3b900aab4be6 admin admin] PATH_INFO: `/v3/projects/01307ac8aa3b4b64a54b2c4999dc4966/users/8a804d65eb584d69a390452f78556a68/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:07.526123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1213d1-e21b-4358-a43d-7bfbc4a39a86 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:07.541449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62439ec-8936-4cff-87f3-3b900aab4be6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01307ac8aa3b4b64a54b2c4999dc4966, user_id=8a804d65eb584d69a390452f78556a68, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:07.545577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5287d22-43cf-4611-b03a-d009d79c3708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:07.546829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287d22-43cf-4611-b03a-d009d79c3708 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:07.548543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287d22-43cf-4611-b03a-d009d79c3708 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:07.549525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62439ec-8936-4cff-87f3-3b900aab4be6 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:07.550377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287d22-43cf-4611-b03a-d009d79c3708 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:07.552862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5287d22-43cf-4611-b03a-d009d79c3708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:07.555399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5287d22-43cf-4611-b03a-d009d79c3708 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:07.595087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 897/1781] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:07 2022] PUT /identity/v3/projects/01307ac8aa3b4b64a54b2c4999dc4966/users/8a804d65eb584d69a390452f78556a68/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:07.596034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62439ec-8936-4cff-87f3-3b900aab4be6 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:07.611246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 885/1782] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:07 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:07.613264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5287d22-43cf-4611-b03a-d009d79c3708 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:07.653313 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5287d22-43cf-4611-b03a-d009d79c3708 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:07.763634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0ae82-c370-41d4-94c5-466e5d4085e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:07.765296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0ae82-c370-41d4-94c5-466e5d4085e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:07.765694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0ae82-c370-41d4-94c5-466e5d4085e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:07.766778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0ae82-c370-41d4-94c5-466e5d4085e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:07.768951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0ae82-c370-41d4-94c5-466e5d4085e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:07.770717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a0ae82-c370-41d4-94c5-466e5d4085e0 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:07.784785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 886/1783] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:07.785760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0ae82-c370-41d4-94c5-466e5d4085e0 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:07.800437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0ae82-c370-41d4-94c5-466e5d4085e0 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:07.805998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c37d3-fe96-426f-8423-83172e4680f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:07.806752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c37d3-fe96-426f-8423-83172e4680f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:07.807237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c37d3-fe96-426f-8423-83172e4680f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:07.807614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c37d3-fe96-426f-8423-83172e4680f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:07.808809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c37d3-fe96-426f-8423-83172e4680f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:07.812823 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c37d3-fe96-426f-8423-83172e4680f4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:07.851606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62439ec-8936-4cff-87f3-3b900aab4be6 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:07.854872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 887/1784] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:07.857332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca7520-4f29-42ed-b22f-58df6fcc1447 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:07.857763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca7520-4f29-42ed-b22f-58df6fcc1447 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:07.858177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca7520-4f29-42ed-b22f-58df6fcc1447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:07.863328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c7ac5-8d4b-43af-a18b-6ba38c84b0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:07.863964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c7ac5-8d4b-43af-a18b-6ba38c84b0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:07.864249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c7ac5-8d4b-43af-a18b-6ba38c84b0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:07.865669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c7ac5-8d4b-43af-a18b-6ba38c84b0f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:07.866334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c7ac5-8d4b-43af-a18b-6ba38c84b0f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:07.872844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c7ac5-8d4b-43af-a18b-6ba38c84b0f2 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:07.925230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:07.954474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 898/1785] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:07 2022] POST /identity/v3/projects => generated 363 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:07.964165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c7ac5-8d4b-43af-a18b-6ba38c84b0f2 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:07.977361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-06ca7520-4f29-42ed-b22f-58df6fcc1447 None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:07.978803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-06ca7520-4f29-42ed-b22f-58df6fcc1447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:08.068513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c7ac5-8d4b-43af-a18b-6ba38c84b0f2 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:08.080011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4543-1b0e-4343-a005-77ff2fd77162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:08.081034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4543-1b0e-4343-a005-77ff2fd77162 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:08.081638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4543-1b0e-4343-a005-77ff2fd77162 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:08.082184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4543-1b0e-4343-a005-77ff2fd77162 admin admin] PATH_INFO: `/v3/projects/01307ac8aa3b4b64a54b2c4999dc4966/users/8a804d65eb584d69a390452f78556a68/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:08.092047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d4543-1b0e-4343-a005-77ff2fd77162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01307ac8aa3b4b64a54b2c4999dc4966, user_id=8a804d65eb584d69a390452f78556a68, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:08.096665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d4543-1b0e-4343-a005-77ff2fd77162 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:08.124935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 899/1786] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:07 2022] PUT /identity/v3/projects/01307ac8aa3b4b64a54b2c4999dc4966/users/8a804d65eb584d69a390452f78556a68/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:08.124935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4543-1b0e-4343-a005-77ff2fd77162 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:08.177389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:08.180675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 888/1787] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:30:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:08.184097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58d665-7992-4686-a64d-29920dc17cb2 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:08.184727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58d665-7992-4686-a64d-29920dc17cb2 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:08.185889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58d665-7992-4686-a64d-29920dc17cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:08.279169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fc58d665-7992-4686-a64d-29920dc17cb2 None None] MFA Rules not processed for user `5bf265cb4af14e2d8815c42288e2a1f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:08.280676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fc58d665-7992-4686-a64d-29920dc17cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:08.290965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4543-1b0e-4343-a005-77ff2fd77162 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:08.305068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a66bea-21b4-4292-b160-1cacc583fc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:08.305699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a66bea-21b4-4292-b160-1cacc583fc7e admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:08.305978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a66bea-21b4-4292-b160-1cacc583fc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:08.306252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a66bea-21b4-4292-b160-1cacc583fc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:08.306905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a66bea-21b4-4292-b160-1cacc583fc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:08.308658 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a66bea-21b4-4292-b160-1cacc583fc7e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:08.353075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 900/1788] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:08.361479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a66bea-21b4-4292-b160-1cacc583fc7e admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:08.397613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a66bea-21b4-4292-b160-1cacc583fc7e admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:08.410462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60237e-d347-4921-a3a2-a9c512253f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:08.421966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60237e-d347-4921-a3a2-a9c512253f94 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:08.422284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60237e-d347-4921-a3a2-a9c512253f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:08.422578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60237e-d347-4921-a3a2-a9c512253f94 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:08.423253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60237e-d347-4921-a3a2-a9c512253f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:08.425276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60237e-d347-4921-a3a2-a9c512253f94 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:08.473609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 889/1789] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:08.481471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae85879-e311-4786-886b-66856b54b840 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:08.481954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae85879-e311-4786-886b-66856b54b840 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:08.482356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae85879-e311-4786-886b-66856b54b840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:08.510422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 901/1790] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:08 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:08.512515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60237e-d347-4921-a3a2-a9c512253f94 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:08.540357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60237e-d347-4921-a3a2-a9c512253f94 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:08.564229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29899ad-6f0f-4703-801c-e2dbb8b77ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:08.565720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29899ad-6f0f-4703-801c-e2dbb8b77ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:08.566175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29899ad-6f0f-4703-801c-e2dbb8b77ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:08.568712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-5ae85879-e311-4786-886b-66856b54b840 None None] MFA Rules not processed for user `8a804d65eb584d69a390452f78556a68`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:08.570111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5ae85879-e311-4786-886b-66856b54b840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:08.573201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29899ad-6f0f-4703-801c-e2dbb8b77ce4 admin admin] PATH_INFO: `/v3/projects/74a98abf867042db8fe688d9bfa70dd1/users/afb8dd749a124e2c807feef2f48ee1b4/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:08.663145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29899ad-6f0f-4703-801c-e2dbb8b77ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a98abf867042db8fe688d9bfa70dd1, user_id=afb8dd749a124e2c807feef2f48ee1b4, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:08.679266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29899ad-6f0f-4703-801c-e2dbb8b77ce4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:08.716786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 902/1791] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:08 2022] PUT /identity/v3/projects/74a98abf867042db8fe688d9bfa70dd1/users/afb8dd749a124e2c807feef2f48ee1b4/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:08.717753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29899ad-6f0f-4703-801c-e2dbb8b77ce4 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:08.788599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 890/1792] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:08.790149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae85879-e311-4786-886b-66856b54b840 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:08.792781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5ae85879-e311-4786-886b-66856b54b840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:08.920314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:08.921684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29899ad-6f0f-4703-801c-e2dbb8b77ce4 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:08.941282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d20d1-dbb2-47a6-8683-b5056dd0e47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:08.944786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d20d1-dbb2-47a6-8683-b5056dd0e47a admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:08.944786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d20d1-dbb2-47a6-8683-b5056dd0e47a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:08.944786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d20d1-dbb2-47a6-8683-b5056dd0e47a admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:08.944786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d20d1-dbb2-47a6-8683-b5056dd0e47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:08.955250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d20d1-dbb2-47a6-8683-b5056dd0e47a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:08.987407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 903/1793] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:08.987407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645da0a-26f7-49c2-b2b4-c8d1b9b8cd59 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:08.987407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645da0a-26f7-49c2-b2b4-c8d1b9b8cd59 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:08.987407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645da0a-26f7-49c2-b2b4-c8d1b9b8cd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:09.074930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0645da0a-26f7-49c2-b2b4-c8d1b9b8cd59 None None] MFA Rules not processed for user `afb8dd749a124e2c807feef2f48ee1b4`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:09.076333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0645da0a-26f7-49c2-b2b4-c8d1b9b8cd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:09.108289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae85879-e311-4786-886b-66856b54b840 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:09.289159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec33e6-6d71-4d73-8201-de605aba6b7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:09.289871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 904/1794] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:09.290644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645da0a-26f7-49c2-b2b4-c8d1b9b8cd59 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:09.290874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec33e6-6d71-4d73-8201-de605aba6b7e service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:09.291365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec33e6-6d71-4d73-8201-de605aba6b7e service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:09.291888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec33e6-6d71-4d73-8201-de605aba6b7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:09.292490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0645da0a-26f7-49c2-b2b4-c8d1b9b8cd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:09.348862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:09.466926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645da0a-26f7-49c2-b2b4-c8d1b9b8cd59 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:09.475251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec33e6-6d71-4d73-8201-de605aba6b7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:09.476945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec33e6-6d71-4d73-8201-de605aba6b7e service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:09.482657 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032601a-f066-4251-bd0d-b904997a019a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:09.483793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032601a-f066-4251-bd0d-b904997a019a service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:09.484082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032601a-f066-4251-bd0d-b904997a019a service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:09.486190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032601a-f066-4251-bd0d-b904997a019a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:09.514839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 891/1795] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:30:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:30:09.517448 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec33e6-6d71-4d73-8201-de605aba6b7e service swift] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:09.542413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec33e6-6d71-4d73-8201-de605aba6b7e service swift] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:09.556673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17693522-3c83-450b-ae07-122b7f935444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:09.557360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17693522-3c83-450b-ae07-122b7f935444 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:09.557635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17693522-3c83-450b-ae07-122b7f935444 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:09.557904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17693522-3c83-450b-ae07-122b7f935444 admin admin] PATH_INFO: `/v3/projects/f6785d2c1b524fb68c4ed995c1851497/users/5f26029e338c4363be5c859060b157c1/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:09.687197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17693522-3c83-450b-ae07-122b7f935444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6785d2c1b524fb68c4ed995c1851497, user_id=5f26029e338c4363be5c859060b157c1, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:09.694359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17693522-3c83-450b-ae07-122b7f935444 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:09.740425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 892/1796] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:09 2022] PUT /identity/v3/projects/f6785d2c1b524fb68c4ed995c1851497/users/5f26029e338c4363be5c859060b157c1/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:09.741892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17693522-3c83-450b-ae07-122b7f935444 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:09.790427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:09.798413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032601a-f066-4251-bd0d-b904997a019a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:09.812055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2032601a-f066-4251-bd0d-b904997a019a service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:09.861254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 905/1797] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:09.863437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032601a-f066-4251-bd0d-b904997a019a service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:09.915636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032601a-f066-4251-bd0d-b904997a019a service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:09.940581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901622-dc58-4d79-a969-ca7907779db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:09.942754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901622-dc58-4d79-a969-ca7907779db0 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:09.943187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901622-dc58-4d79-a969-ca7907779db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:09.944323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901622-dc58-4d79-a969-ca7907779db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:09.945163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5901622-dc58-4d79-a969-ca7907779db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:09.952378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5901622-dc58-4d79-a969-ca7907779db0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:09.988697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17693522-3c83-450b-ae07-122b7f935444 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:10.003049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 906/1798] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:10.008390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901622-dc58-4d79-a969-ca7907779db0 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:10.032742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7f5b5-044d-4a09-9402-c7ed39f70d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:10.033738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7f5b5-044d-4a09-9402-c7ed39f70d26 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:10.033936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7f5b5-044d-4a09-9402-c7ed39f70d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:10.034272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7f5b5-044d-4a09-9402-c7ed39f70d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:10.041156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7f5b5-044d-4a09-9402-c7ed39f70d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:10.052689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7f5b5-044d-4a09-9402-c7ed39f70d26 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:10.062335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901622-dc58-4d79-a969-ca7907779db0 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:10.093647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904397b-700f-4a33-b4c9-ae6e3da623eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:10.099173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904397b-700f-4a33-b4c9-ae6e3da623eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:10.099638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904397b-700f-4a33-b4c9-ae6e3da623eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:10.100068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904397b-700f-4a33-b4c9-ae6e3da623eb admin admin] PATH_INFO: `/v3/projects/f6785d2c1b524fb68c4ed995c1851497/users/5f26029e338c4363be5c859060b157c1/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:10.103357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904397b-700f-4a33-b4c9-ae6e3da623eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6785d2c1b524fb68c4ed995c1851497, user_id=5f26029e338c4363be5c859060b157c1, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:10.115293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904397b-700f-4a33-b4c9-ae6e3da623eb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:10.160678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 907/1799] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:10 2022] PUT /identity/v3/projects/f6785d2c1b524fb68c4ed995c1851497/users/5f26029e338c4363be5c859060b157c1/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:10.166612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904397b-700f-4a33-b4c9-ae6e3da623eb admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:10.174594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0904397b-700f-4a33-b4c9-ae6e3da623eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:10.200767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 893/1800] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:09 2022] POST /identity/v3/projects => generated 385 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:10.210268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781c9fe-ef0a-46ab-95af-b9121b81ac58 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:10.211055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781c9fe-ef0a-46ab-95af-b9121b81ac58 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:10.213576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781c9fe-ef0a-46ab-95af-b9121b81ac58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:10.339866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-9781c9fe-ef0a-46ab-95af-b9121b81ac58 None None] MFA Rules not processed for user `5f26029e338c4363be5c859060b157c1`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:10.341952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9781c9fe-ef0a-46ab-95af-b9121b81ac58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:10.409055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904397b-700f-4a33-b4c9-ae6e3da623eb admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:10.485920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223976c-12d0-440c-bebc-1c95c78dc16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:10.486647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223976c-12d0-440c-bebc-1c95c78dc16b service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:10.486993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223976c-12d0-440c-bebc-1c95c78dc16b service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:10.487469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223976c-12d0-440c-bebc-1c95c78dc16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:10.512094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:10.524443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 894/1801] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:10.525394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781c9fe-ef0a-46ab-95af-b9121b81ac58 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:10.666104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223976c-12d0-440c-bebc-1c95c78dc16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:10.673699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223976c-12d0-440c-bebc-1c95c78dc16b service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:10.716754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 908/1802] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:10.722951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223976c-12d0-440c-bebc-1c95c78dc16b service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:10.730932 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781c9fe-ef0a-46ab-95af-b9121b81ac58 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:10.742524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223976c-12d0-440c-bebc-1c95c78dc16b service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:10.743578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceb313-636d-4d19-9e58-abb41cb9a469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:10.744296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceb313-636d-4d19-9e58-abb41cb9a469 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:10.745162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceb313-636d-4d19-9e58-abb41cb9a469 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:10.745497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceb313-636d-4d19-9e58-abb41cb9a469 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:10.747096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceb313-636d-4d19-9e58-abb41cb9a469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:10.750722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ceb313-636d-4d19-9e58-abb41cb9a469 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:10.758820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9229a4f-bc50-4390-b2cb-0430ccd50eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:10.759818 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9229a4f-bc50-4390-b2cb-0430ccd50eac service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:10.760302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9229a4f-bc50-4390-b2cb-0430ccd50eac service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:10.760795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9229a4f-bc50-4390-b2cb-0430ccd50eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:10.796243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 895/1803] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:10 2022] POST /identity/v3/users => generated 299 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:10.804149 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9229a4f-bc50-4390-b2cb-0430ccd50eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:10.806839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceb313-636d-4d19-9e58-abb41cb9a469 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:10.818447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9229a4f-bc50-4390-b2cb-0430ccd50eac service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:10.836303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ceb313-636d-4d19-9e58-abb41cb9a469 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:10.842859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 909/1804] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:10.864755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b636c6-62ea-4294-b99c-51666abe41ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:10.865379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b636c6-62ea-4294-b99c-51666abe41ca admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:10.865660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b636c6-62ea-4294-b99c-51666abe41ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:10.870819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b636c6-62ea-4294-b99c-51666abe41ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:10.871642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b636c6-62ea-4294-b99c-51666abe41ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:10.873342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b636c6-62ea-4294-b99c-51666abe41ca admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:10.928912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9229a4f-bc50-4390-b2cb-0430ccd50eac service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:10.930867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 896/1805] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:10.980668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9229a4f-bc50-4390-b2cb-0430ccd50eac service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:11.021265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67410a4a-d9d1-471f-b892-e12d3c66ef34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:11.022176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67410a4a-d9d1-471f-b892-e12d3c66ef34 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:11.024425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67410a4a-d9d1-471f-b892-e12d3c66ef34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:11.026038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67410a4a-d9d1-471f-b892-e12d3c66ef34 admin admin] PATH_INFO: `/v3/projects/6b126e1169ce4b709c785dc40314b203/users/97c2cb482c6b469e8eb25637187dce27/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:11.152026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67410a4a-d9d1-471f-b892-e12d3c66ef34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b126e1169ce4b709c785dc40314b203, user_id=97c2cb482c6b469e8eb25637187dce27, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:11.165555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67410a4a-d9d1-471f-b892-e12d3c66ef34 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:11.233405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 910/1806] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:10 2022] PUT /identity/v3/projects/6b126e1169ce4b709c785dc40314b203/users/97c2cb482c6b469e8eb25637187dce27/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:11.237419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b636c6-62ea-4294-b99c-51666abe41ca admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:11.252937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b636c6-62ea-4294-b99c-51666abe41ca admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:11.335807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:11.443013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67410a4a-d9d1-471f-b892-e12d3c66ef34 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:11.445854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-67410a4a-d9d1-471f-b892-e12d3c66ef34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:11.508247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aecab-aa68-4998-bd1b-e3bd31bdc734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:11.509210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aecab-aa68-4998-bd1b-e3bd31bdc734 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:11.509969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aecab-aa68-4998-bd1b-e3bd31bdc734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:11.510268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aecab-aa68-4998-bd1b-e3bd31bdc734 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:11.511033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aecab-aa68-4998-bd1b-e3bd31bdc734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:11.514541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aecab-aa68-4998-bd1b-e3bd31bdc734 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:11.550466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 897/1807] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:11.559087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aecab-aa68-4998-bd1b-e3bd31bdc734 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:11.587509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aecab-aa68-4998-bd1b-e3bd31bdc734 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:11.628308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd05876-56a5-4efe-836f-5889d86277de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:11.631269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd05876-56a5-4efe-836f-5889d86277de admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:11.633928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd05876-56a5-4efe-836f-5889d86277de admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:11.635513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd05876-56a5-4efe-836f-5889d86277de admin admin] PATH_INFO: `/v3/projects/6b126e1169ce4b709c785dc40314b203/users/97c2cb482c6b469e8eb25637187dce27/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:11.654688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd05876-56a5-4efe-836f-5889d86277de admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b126e1169ce4b709c785dc40314b203, user_id=97c2cb482c6b469e8eb25637187dce27, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:11.677086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd05876-56a5-4efe-836f-5889d86277de admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:11.720151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67410a4a-d9d1-471f-b892-e12d3c66ef34 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:11.725769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:11.726853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:11.727353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:11.727858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] PATH_INFO: `/v3/users/44b6b4c2bb804a8e9c1f049fed14bde8` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:11.732835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b6b4c2bb804a8e9c1f049fed14bde8)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:11.741099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:11.775776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 898/1808] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:11 2022] PUT /identity/v3/projects/6b126e1169ce4b709c785dc40314b203/users/97c2cb482c6b469e8eb25637187dce27/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:11.799401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4e92c-885f-4f05-870d-1c0155c1d07b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:11.806506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4e92c-885f-4f05-870d-1c0155c1d07b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:11.808058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4e92c-885f-4f05-870d-1c0155c1d07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:11.918978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-a2e4e92c-885f-4f05-870d-1c0155c1d07b None None] MFA Rules not processed for user `97c2cb482c6b469e8eb25637187dce27`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:11.924181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a2e4e92c-885f-4f05-870d-1c0155c1d07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:11.944523 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b6b4c2bb804a8e9c1f049fed14bde8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:11.946114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b6b4c2bb804a8e9c1f049fed14bde8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:11.980231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b6b4c2bb804a8e9c1f049fed14bde8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:11.994016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b6b4c2bb804a8e9c1f049fed14bde8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:11.995974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b6b4c2bb804a8e9c1f049fed14bde8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:12.030545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:12.041961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 911/1809] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:11 2022] DELETE /identity/v3/users/44b6b4c2bb804a8e9c1f049fed14bde8 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:12.043025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:12.053074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:12.179117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 899/1810] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:12.195905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4143811-f59d-427c-83ea-cd44502413f7 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:12.202452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4143811-f59d-427c-83ea-cd44502413f7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:12.203218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4143811-f59d-427c-83ea-cd44502413f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:12.224036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33ef00-3a4e-480b-bf9b-11289650e91b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:12.243697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:12.244362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:12.244672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:12.244950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] PATH_INFO: `/v3/users/6dcc16a91c094bc7b7636d978e069219` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:12.247745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dcc16a91c094bc7b7636d978e069219)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:12.249752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:12.320846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e4143811-f59d-427c-83ea-cd44502413f7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:12.325493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e4143811-f59d-427c-83ea-cd44502413f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:12.371141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 900/1811] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:12.372376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4143811-f59d-427c-83ea-cd44502413f7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:12.384871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e4143811-f59d-427c-83ea-cd44502413f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:12.444583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dcc16a91c094bc7b7636d978e069219'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:12.445914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dcc16a91c094bc7b7636d978e069219'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:12.471512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dcc16a91c094bc7b7636d978e069219'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:12.484092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dcc16a91c094bc7b7636d978e069219'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:12.487063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e7f3589d-71a1-4822-9065-4450245aee4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dcc16a91c094bc7b7636d978e069219'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:12.520595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 912/1812] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:12 2022] DELETE /identity/v3/users/6dcc16a91c094bc7b7636d978e069219 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:12.522342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ca43f-0689-4b6a-afdb-1103a589f09f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:12.522636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ca43f-0689-4b6a-afdb-1103a589f09f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:12.523269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ca43f-0689-4b6a-afdb-1103a589f09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:12.563025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b83ca43f-0689-4b6a-afdb-1103a589f09f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:12.570977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b83ca43f-0689-4b6a-afdb-1103a589f09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:12.657039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4143811-f59d-427c-83ea-cd44502413f7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:12.777309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:12.815843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 913/1813] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:12.816882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ca43f-0689-4b6a-afdb-1103a589f09f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:12.829530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b83ca43f-0689-4b6a-afdb-1103a589f09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:12.853309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3978ae-4d3e-4427-ba56-68982ed6d409 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:12.861869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3978ae-4d3e-4427-ba56-68982ed6d409 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:12.861869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3978ae-4d3e-4427-ba56-68982ed6d409 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:12.861869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3978ae-4d3e-4427-ba56-68982ed6d409 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:12.863565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0c3978ae-4d3e-4427-ba56-68982ed6d409 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:12.882629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:12.976000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ca43f-0689-4b6a-afdb-1103a589f09f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:12.979626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c94a13-9b39-42ac-ab28-70f26f495718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:12.980251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c94a13-9b39-42ac-ab28-70f26f495718 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:12.980553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c94a13-9b39-42ac-ab28-70f26f495718 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:12.980830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c94a13-9b39-42ac-ab28-70f26f495718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:12.982661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a2c94a13-9b39-42ac-ab28-70f26f495718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:12.995691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c94a13-9b39-42ac-ab28-70f26f495718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:12.996879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c94a13-9b39-42ac-ab28-70f26f495718 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:13.029289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 914/1814] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:13.030515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c94a13-9b39-42ac-ab28-70f26f495718 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:13.041320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c94a13-9b39-42ac-ab28-70f26f495718 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:13.074733 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1c452-67b7-4d97-a352-48de7825da9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:13.075735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1c452-67b7-4d97-a352-48de7825da9f service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:13.076169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1c452-67b7-4d97-a352-48de7825da9f service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:13.076618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1c452-67b7-4d97-a352-48de7825da9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:13.096375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1c452-67b7-4d97-a352-48de7825da9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:13.098055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1c452-67b7-4d97-a352-48de7825da9f service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:13.120783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 915/1815] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:13.131699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3978ae-4d3e-4427-ba56-68982ed6d409 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:13.133683 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3978ae-4d3e-4427-ba56-68982ed6d409 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:13.170482 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 901/1816] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 798 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:14.261958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813a13-77e8-4fee-bc92-299008a0b34c None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:14.262936 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813a13-77e8-4fee-bc92-299008a0b34c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:14.263416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813a13-77e8-4fee-bc92-299008a0b34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:14.329043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0734542-0fe0-4d1b-a533-8bc4914b7cc9 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:14.330341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0734542-0fe0-4d1b-a533-8bc4914b7cc9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:14.330842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0734542-0fe0-4d1b-a533-8bc4914b7cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:14.352304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3b813a13-77e8-4fee-bc92-299008a0b34c None None] MFA Rules not processed for user `cdf97e9e0340441c87316ce03d46cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:14.353813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3b813a13-77e8-4fee-bc92-299008a0b34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:14.371112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-f0734542-0fe0-4d1b-a533-8bc4914b7cc9 None None] MFA Rules not processed for user `ecf29db866eb43fa954b0d440577f45f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:14.372216 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 916/1817] 10.222.0.30 () {58 vars in 1015 bytes} [Thu Sep 1 00:30:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:14.386357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f0734542-0fe0-4d1b-a533-8bc4914b7cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:14.433031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813a13-77e8-4fee-bc92-299008a0b34c None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:14.435528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3b813a13-77e8-4fee-bc92-299008a0b34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:14.474243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813a13-77e8-4fee-bc92-299008a0b34c None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:14.512375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:14.513567 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:14.516153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:14.518276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] PATH_INFO: `/v3/users/2b82b880d82945478ffa8d811341d095` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:14.523052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b82b880d82945478ffa8d811341d095)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:14.531965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:14.626715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 902/1818] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:14.682962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b82b880d82945478ffa8d811341d095'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:14.683214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b82b880d82945478ffa8d811341d095'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:14.699470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b82b880d82945478ffa8d811341d095'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:14.711051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0734542-0fe0-4d1b-a533-8bc4914b7cc9 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:14.712743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f0734542-0fe0-4d1b-a533-8bc4914b7cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:14.738866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b82b880d82945478ffa8d811341d095'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:14.741048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bd778006-7813-4102-b0ce-9c1b6ee45689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b82b880d82945478ffa8d811341d095'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:14.743735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0734542-0fe0-4d1b-a533-8bc4914b7cc9 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:14.765282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:14.774205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:14.776173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:14.779310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] PATH_INFO: `/v3/users/afb8dd749a124e2c807feef2f48ee1b4` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:14.785446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb8dd749a124e2c807feef2f48ee1b4)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:14.790347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:14.823888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 917/1819] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:14 2022] DELETE /identity/v3/users/2b82b880d82945478ffa8d811341d095 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:14.832077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76395779-b869-4487-a8b1-45e8b57965df None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:14.832394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76395779-b869-4487-a8b1-45e8b57965df None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:14.834356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76395779-b869-4487-a8b1-45e8b57965df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:14.920180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-76395779-b869-4487-a8b1-45e8b57965df None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:14.923055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-76395779-b869-4487-a8b1-45e8b57965df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:14.961539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb8dd749a124e2c807feef2f48ee1b4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:14.962307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb8dd749a124e2c807feef2f48ee1b4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:15.005849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb8dd749a124e2c807feef2f48ee1b4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:15.025294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb8dd749a124e2c807feef2f48ee1b4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:15.044864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb8dd749a124e2c807feef2f48ee1b4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:15.048840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:15.054316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 903/1820] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:14 2022] DELETE /identity/v3/users/afb8dd749a124e2c807feef2f48ee1b4 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:15.058361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:15.062054 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:15.096692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 918/1821] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:15.096692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76395779-b869-4487-a8b1-45e8b57965df None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:15.099811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-76395779-b869-4487-a8b1-45e8b57965df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:15.158623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:15.294250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76395779-b869-4487-a8b1-45e8b57965df None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:15.307075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8e57-8450-4bad-9916-ddbcaae9c7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:15.307939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8e57-8450-4bad-9916-ddbcaae9c7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:15.308370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8e57-8450-4bad-9916-ddbcaae9c7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:15.308886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8e57-8450-4bad-9916-ddbcaae9c7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:15.309120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b236047-a542-4570-9def-13d2f3767fc2 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:15.324185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ee8e8e57-8450-4bad-9916-ddbcaae9c7b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:15.327580 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30ba0d-2828-423f-bccb-e16179125b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:15.329207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30ba0d-2828-423f-bccb-e16179125b57 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:15.329798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30ba0d-2828-423f-bccb-e16179125b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:15.329798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30ba0d-2828-423f-bccb-e16179125b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:15.361866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8e57-8450-4bad-9916-ddbcaae9c7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:15.365989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8e57-8450-4bad-9916-ddbcaae9c7b3 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:15.383462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30ba0d-2828-423f-bccb-e16179125b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:15.385052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30ba0d-2828-423f-bccb-e16179125b57 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:15.388305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 919/1822] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:15.389977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11560a11-2bf1-414c-8f10-42a620009772 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:15.390265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11560a11-2bf1-414c-8f10-42a620009772 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:15.397626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11560a11-2bf1-414c-8f10-42a620009772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:15.400189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 904/1823] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:15.401094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30ba0d-2828-423f-bccb-e16179125b57 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:15.420087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30ba0d-2828-423f-bccb-e16179125b57 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:15.429442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-11560a11-2bf1-414c-8f10-42a620009772 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:15.430797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-11560a11-2bf1-414c-8f10-42a620009772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:15.433174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f25f9-b83e-462d-8728-f9012d54e205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:15.440766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f25f9-b83e-462d-8728-f9012d54e205 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:15.441391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f25f9-b83e-462d-8728-f9012d54e205 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:15.442006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f25f9-b83e-462d-8728-f9012d54e205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:15.592852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 920/1824] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:15.612721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f25f9-b83e-462d-8728-f9012d54e205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:15.616793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f25f9-b83e-462d-8728-f9012d54e205 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:15.631682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 905/1825] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:15.733886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11560a11-2bf1-414c-8f10-42a620009772 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:15.750145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11560a11-2bf1-414c-8f10-42a620009772 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:15.788611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63bd67-7328-4cea-bcfd-6a6638246bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:15.790607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63bd67-7328-4cea-bcfd-6a6638246bce service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:15.790698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63bd67-7328-4cea-bcfd-6a6638246bce service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:15.790958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63bd67-7328-4cea-bcfd-6a6638246bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:15.814771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63bd67-7328-4cea-bcfd-6a6638246bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:15.825080 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d63bd67-7328-4cea-bcfd-6a6638246bce service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:15.847742 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 921/1826] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:16.609754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f25f9-b83e-462d-8728-f9012d54e205 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:16.611818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fe6f25f9-b83e-462d-8728-f9012d54e205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:16.639200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f25f9-b83e-462d-8728-f9012d54e205 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:16.652251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:16.652993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:16.653276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:16.653544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] PATH_INFO: `/v3/projects/d2cb59a42b584d07915d5b1ae4e9e3b6` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:16.661260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2cb59a42b584d07915d5b1ae4e9e3b6)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:16.681099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:16.796311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2cb59a42b584d07915d5b1ae4e9e3b6', 'request_id': 'req-5c37d33a-d325-41c1-9674-94ea8ea63644'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:16.812321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2cb59a42b584d07915d5b1ae4e9e3b6', 'request_id': 'req-5c37d33a-d325-41c1-9674-94ea8ea63644'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:16.826925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2cb59a42b584d07915d5b1ae4e9e3b6', 'request_id': 'req-5c37d33a-d325-41c1-9674-94ea8ea63644'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:16.864969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] The token cache is being invalidate because project d2cb59a42b584d07915d5b1ae4e9e3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:16.865270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:16.865954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:16.867780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 906/1827] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:16 2022] DELETE /identity/v3/projects/d2cb59a42b584d07915d5b1ae4e9e3b6 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:16.872842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63bd67-7328-4cea-bcfd-6a6638246bce service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:16.884309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d63bd67-7328-4cea-bcfd-6a6638246bce service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:16.936830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:16.937255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:17.019362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:17.149095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:17.151128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:17.152103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:17.152586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] PATH_INFO: `/v3/users/5f26029e338c4363be5c859060b157c1` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:17.154943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f26029e338c4363be5c859060b157c1)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:17.160317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:17.210983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c37d33a-d325-41c1-9674-94ea8ea63644 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:17.245315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:17.246143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:17.246525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:17.246915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] PATH_INFO: `/v3/projects/64544810ae6542b798a4253a3e963067` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:17.257757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=64544810ae6542b798a4253a3e963067)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:17.262335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:17.301702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:17.305972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f26029e338c4363be5c859060b157c1'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.309710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f26029e338c4363be5c859060b157c1'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.336408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f26029e338c4363be5c859060b157c1'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.351556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f26029e338c4363be5c859060b157c1'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.353550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6b15a7d3-429d-4d19-9323-3c3d99c97b38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f26029e338c4363be5c859060b157c1'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.416662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64544810ae6542b798a4253a3e963067', 'request_id': 'req-a0d229c2-bb96-4b0a-8288-23757f0560ce'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.423008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 922/1828] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:16 2022] DELETE /identity/v3/users/5f26029e338c4363be5c859060b157c1 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:17.441425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64544810ae6542b798a4253a3e963067', 'request_id': 'req-a0d229c2-bb96-4b0a-8288-23757f0560ce'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.453869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64544810ae6542b798a4253a3e963067', 'request_id': 'req-a0d229c2-bb96-4b0a-8288-23757f0560ce'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.477430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] The token cache is being invalidate because project 64544810ae6542b798a4253a3e963067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:17.477731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.478410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a0d229c2-bb96-4b0a-8288-23757f0560ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:17.484830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 907/1829] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:16 2022] DELETE /identity/v3/projects/64544810ae6542b798a4253a3e963067 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:17.485731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd098f0-23a3-42c0-ba51-1d42e4cf5263 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:17.486163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd098f0-23a3-42c0-ba51-1d42e4cf5263 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:17.486362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd098f0-23a3-42c0-ba51-1d42e4cf5263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:17.515342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5bd098f0-23a3-42c0-ba51-1d42e4cf5263 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:17.516731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5bd098f0-23a3-42c0-ba51-1d42e4cf5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:17.608676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 923/1830] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:17.617462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0d6b8-8751-4dec-a5b7-6e3a6abaa16b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:17.618461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0d6b8-8751-4dec-a5b7-6e3a6abaa16b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:17.618950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0d6b8-8751-4dec-a5b7-6e3a6abaa16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:17.661111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1fb0d6b8-8751-4dec-a5b7-6e3a6abaa16b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:17.663009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1fb0d6b8-8751-4dec-a5b7-6e3a6abaa16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:17.683774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 908/1831] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:17.688649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd098f0-23a3-42c0-ba51-1d42e4cf5263 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:17.713371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd098f0-23a3-42c0-ba51-1d42e4cf5263 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:17.732867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3de09-53ab-490f-831b-f88abf0602ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:17.733878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3de09-53ab-490f-831b-f88abf0602ae admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:17.735557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3de09-53ab-490f-831b-f88abf0602ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:17.736055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3de09-53ab-490f-831b-f88abf0602ae admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:17.736968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3de09-53ab-490f-831b-f88abf0602ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:17.737478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3de09-53ab-490f-831b-f88abf0602ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:17.741343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3de09-53ab-490f-831b-f88abf0602ae admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:17.778078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 924/1832] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:17.784098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0d6b8-8751-4dec-a5b7-6e3a6abaa16b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:17.807425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0d6b8-8751-4dec-a5b7-6e3a6abaa16b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:17.817599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fb5d4-5969-428c-81b3-adbec7f914af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:17.818615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fb5d4-5969-428c-81b3-adbec7f914af admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:17.821630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fb5d4-5969-428c-81b3-adbec7f914af admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:17.822353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fb5d4-5969-428c-81b3-adbec7f914af admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:17.823406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fb5d4-5969-428c-81b3-adbec7f914af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:17.826563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fb5d4-5969-428c-81b3-adbec7f914af admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:17.890383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 909/1833] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:17 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:17.893948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3de09-53ab-490f-831b-f88abf0602ae admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:17.945055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3de09-53ab-490f-831b-f88abf0602ae admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:18.082858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd4c6d-fc84-4162-8f5c-20e85696a6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:18.085594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd4c6d-fc84-4162-8f5c-20e85696a6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:18.086042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd4c6d-fc84-4162-8f5c-20e85696a6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:18.086325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd4c6d-fc84-4162-8f5c-20e85696a6d3 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:18.087242 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd4c6d-fc84-4162-8f5c-20e85696a6d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:18.090348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd4c6d-fc84-4162-8f5c-20e85696a6d3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:18.141852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 925/1834] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:17 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:30:18.153844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fb5d4-5969-428c-81b3-adbec7f914af admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:18.175062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fb5d4-5969-428c-81b3-adbec7f914af admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:18.191372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f6035-79ab-4d85-94e0-7b8338a3d3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:18.193057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f6035-79ab-4d85-94e0-7b8338a3d3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:18.193057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f6035-79ab-4d85-94e0-7b8338a3d3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:18.193057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f6035-79ab-4d85-94e0-7b8338a3d3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:18.193377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f6035-79ab-4d85-94e0-7b8338a3d3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:18.195162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f6035-79ab-4d85-94e0-7b8338a3d3a1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:18.221664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 910/1835] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:18.225775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd4c6d-fc84-4162-8f5c-20e85696a6d3 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:18.241969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f6035-79ab-4d85-94e0-7b8338a3d3a1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:18.245381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-709f6035-79ab-4d85-94e0-7b8338a3d3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:18.252473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd4c6d-fc84-4162-8f5c-20e85696a6d3 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:18.279404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f161a9-6544-4464-a011-168a2c6c0e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:18.280101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f161a9-6544-4464-a011-168a2c6c0e7c admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:18.280382 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f161a9-6544-4464-a011-168a2c6c0e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:18.280706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f161a9-6544-4464-a011-168a2c6c0e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:18.281505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f161a9-6544-4464-a011-168a2c6c0e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:18.283313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f161a9-6544-4464-a011-168a2c6c0e7c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:18.287945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f6035-79ab-4d85-94e0-7b8338a3d3a1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:18.303871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:18.305764 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:18.306528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:18.307017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] PATH_INFO: `/v3/projects/74a98abf867042db8fe688d9bfa70dd1` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:18.308424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a98abf867042db8fe688d9bfa70dd1)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:18.310920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:18.325363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 926/1836] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:18.333743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f161a9-6544-4464-a011-168a2c6c0e7c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:18.366049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f161a9-6544-4464-a011-168a2c6c0e7c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:18.381557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46b46b-10a5-4265-b9c3-52b62a69179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:18.382421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46b46b-10a5-4265-b9c3-52b62a69179c admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:18.382779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46b46b-10a5-4265-b9c3-52b62a69179c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:18.383153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46b46b-10a5-4265-b9c3-52b62a69179c admin admin] PATH_INFO: `/v3/projects/0b9b4ab5a7e64c6fb3d414db8c2d0b9d/users/ccc1163b3ca542ae81af1a9e069987d3/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:18.424175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a98abf867042db8fe688d9bfa70dd1', 'request_id': 'req-5a7428e6-e88b-4111-b4aa-d88b471df8ce'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:18.425611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46b46b-10a5-4265-b9c3-52b62a69179c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9b4ab5a7e64c6fb3d414db8c2d0b9d, user_id=ccc1163b3ca542ae81af1a9e069987d3, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:18.428372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46b46b-10a5-4265-b9c3-52b62a69179c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:18.451341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a98abf867042db8fe688d9bfa70dd1', 'request_id': 'req-5a7428e6-e88b-4111-b4aa-d88b471df8ce'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:18.469735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 927/1837] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:18 2022] PUT /identity/v3/projects/0b9b4ab5a7e64c6fb3d414db8c2d0b9d/users/ccc1163b3ca542ae81af1a9e069987d3/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:18.471400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b1550-d58b-402b-be3b-b916217f0442 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:18.471713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b1550-d58b-402b-be3b-b916217f0442 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:18.471988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b1550-d58b-402b-be3b-b916217f0442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:18.475516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a98abf867042db8fe688d9bfa70dd1', 'request_id': 'req-5a7428e6-e88b-4111-b4aa-d88b471df8ce'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:18.483038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:18.501496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:18.504547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] The token cache is being invalidate because project 74a98abf867042db8fe688d9bfa70dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:18.504960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:18.506417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:18.514450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 911/1838] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:18 2022] DELETE /identity/v3/projects/74a98abf867042db8fe688d9bfa70dd1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:18.519618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:18.521507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:18.567482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-953b1550-d58b-402b-be3b-b916217f0442 None None] MFA Rules not processed for user `ccc1163b3ca542ae81af1a9e069987d3`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:18.569273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-953b1550-d58b-402b-be3b-b916217f0442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:18.649746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7428e6-e88b-4111-b4aa-d88b471df8ce admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:18.784098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 928/1839] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:18.790275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b1550-d58b-402b-be3b-b916217f0442 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:18.803654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-953b1550-d58b-402b-be3b-b916217f0442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:18.809866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:18.824625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:18.825025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:18.825348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] PATH_INFO: `/v3/users/97c2cb482c6b469e8eb25637187dce27` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:18.826869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c2cb482c6b469e8eb25637187dce27)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:18.848749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:18.867868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b1550-d58b-402b-be3b-b916217f0442 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:18.886285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e4e6-000b-47a1-8ba8-f59b86c234c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:18.888124 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e4e6-000b-47a1-8ba8-f59b86c234c8 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:18.889401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e4e6-000b-47a1-8ba8-f59b86c234c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:18.892917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e4e6-000b-47a1-8ba8-f59b86c234c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:18.893858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621e4e6-000b-47a1-8ba8-f59b86c234c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:18.896203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621e4e6-000b-47a1-8ba8-f59b86c234c8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:18.969843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c2cb482c6b469e8eb25637187dce27'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:18.970592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c2cb482c6b469e8eb25637187dce27'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:18.971663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 929/1840] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:18 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:18.976136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172962-9c67-4e8e-9676-a4206c8849e3 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:18.976463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172962-9c67-4e8e-9676-a4206c8849e3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:18.976840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172962-9c67-4e8e-9676-a4206c8849e3 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:18.978526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 930/1841] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:30:18.979404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172962-9c67-4e8e-9676-a4206c8849e3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:19.014236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c2cb482c6b469e8eb25637187dce27'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:19.034601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c2cb482c6b469e8eb25637187dce27'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:19.054309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5db0ce0-e969-420a-b792-f8e35cbe0df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c2cb482c6b469e8eb25637187dce27'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:19.061934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 912/1842] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:18 2022] DELETE /identity/v3/users/97c2cb482c6b469e8eb25637187dce27 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:19.095653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80ad8d-5ba4-4fb6-bdbf-d8e4c1065ce6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:19.096315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80ad8d-5ba4-4fb6-bdbf-d8e4c1065ce6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:19.096315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80ad8d-5ba4-4fb6-bdbf-d8e4c1065ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:19.139081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-aa80ad8d-5ba4-4fb6-bdbf-d8e4c1065ce6 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:19.144199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-aa80ad8d-5ba4-4fb6-bdbf-d8e4c1065ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:19.178032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:19.195168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172962-9c67-4e8e-9676-a4206c8849e3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:19.334998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 913/1843] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:30:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:19.362142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e6b48-d858-4820-882e-8f7d32d8e436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:19.362768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e6b48-d858-4820-882e-8f7d32d8e436 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:19.363077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e6b48-d858-4820-882e-8f7d32d8e436 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:19.363324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e6b48-d858-4820-882e-8f7d32d8e436 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:19.363965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e6b48-d858-4820-882e-8f7d32d8e436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:19.365933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e6b48-d858-4820-882e-8f7d32d8e436 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:19.432540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 931/1844] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:18 2022] POST /identity/v3/users => generated 289 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:19.435104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80ad8d-5ba4-4fb6-bdbf-d8e4c1065ce6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:19.436750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-aa80ad8d-5ba4-4fb6-bdbf-d8e4c1065ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:19.477962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80ad8d-5ba4-4fb6-bdbf-d8e4c1065ce6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:19.489347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae892b-28e8-4cc5-9d95-4a37b6cc2397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:19.490031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae892b-28e8-4cc5-9d95-4a37b6cc2397 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:19.490351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae892b-28e8-4cc5-9d95-4a37b6cc2397 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:19.490706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae892b-28e8-4cc5-9d95-4a37b6cc2397 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:19.491518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae892b-28e8-4cc5-9d95-4a37b6cc2397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:19.503819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae892b-28e8-4cc5-9d95-4a37b6cc2397 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:19.524856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 914/1845] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:19.527637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e6b48-d858-4820-882e-8f7d32d8e436 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:19.530163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4c1e6b48-d858-4820-882e-8f7d32d8e436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:19.536178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae892b-28e8-4cc5-9d95-4a37b6cc2397 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:19.553796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e6b48-d858-4820-882e-8f7d32d8e436 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:19.559957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae892b-28e8-4cc5-9d95-4a37b6cc2397 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:19.566170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:19.568052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:19.568932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:19.569719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] PATH_INFO: `/v3/projects/65376d442da84facae3d0473b04a4b3e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:19.572925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65376d442da84facae3d0473b04a4b3e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:19.573921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44152072-ef57-4916-be17-86692f807a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:19.576285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:19.579164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44152072-ef57-4916-be17-86692f807a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:19.579724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44152072-ef57-4916-be17-86692f807a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:19.580286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44152072-ef57-4916-be17-86692f807a6d admin admin] PATH_INFO: `/v3/projects/7f57144b0ee84a34bd0b68ffcb97730e/users/75774ee6cdcc4c2698731b096605da38/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:19.590123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:19.631847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44152072-ef57-4916-be17-86692f807a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f57144b0ee84a34bd0b68ffcb97730e, user_id=75774ee6cdcc4c2698731b096605da38, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:19.636325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44152072-ef57-4916-be17-86692f807a6d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:19.669571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 915/1846] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:19 2022] PUT /identity/v3/projects/7f57144b0ee84a34bd0b68ffcb97730e/users/75774ee6cdcc4c2698731b096605da38/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:19.675054 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44152072-ef57-4916-be17-86692f807a6d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:19.713709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65376d442da84facae3d0473b04a4b3e', 'request_id': 'req-5c3bcb99-ea99-4867-a46a-828f55fa9576'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:19.724057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65376d442da84facae3d0473b04a4b3e', 'request_id': 'req-5c3bcb99-ea99-4867-a46a-828f55fa9576'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:19.731876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65376d442da84facae3d0473b04a4b3e', 'request_id': 'req-5c3bcb99-ea99-4867-a46a-828f55fa9576'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:19.766041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] The token cache is being invalidate because project 65376d442da84facae3d0473b04a4b3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:19.766527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:19.767810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:19.774380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 932/1847] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:19 2022] DELETE /identity/v3/projects/65376d442da84facae3d0473b04a4b3e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:19.775792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:19.778460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:19.920521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44152072-ef57-4916-be17-86692f807a6d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:20.000446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3bcb99-ea99-4867-a46a-828f55fa9576 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:20.012201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:20.012883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:20.013152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:20.013513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] PATH_INFO: `/v3/users/8a804d65eb584d69a390452f78556a68` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:20.015711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a804d65eb584d69a390452f78556a68)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:20.017970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:20.086895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994abd3-4ba5-431f-8bd8-e4979b6c8cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:20.094200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994abd3-4ba5-431f-8bd8-e4979b6c8cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:20.094200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994abd3-4ba5-431f-8bd8-e4979b6c8cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:20.094200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994abd3-4ba5-431f-8bd8-e4979b6c8cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:20.094200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994abd3-4ba5-431f-8bd8-e4979b6c8cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:20.094200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994abd3-4ba5-431f-8bd8-e4979b6c8cd6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:20.123710 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 916/1848] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:20.123710 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994abd3-4ba5-431f-8bd8-e4979b6c8cd6 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:20.130572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7994abd3-4ba5-431f-8bd8-e4979b6c8cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:20.146962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a804d65eb584d69a390452f78556a68'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:20.149499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a804d65eb584d69a390452f78556a68'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:20.165269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a804d65eb584d69a390452f78556a68'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:20.181991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a804d65eb584d69a390452f78556a68'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:20.184735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a804d65eb584d69a390452f78556a68'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:20.188072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:20.273765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 933/1849] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:19 2022] DELETE /identity/v3/users/8a804d65eb584d69a390452f78556a68 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:20.274790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:20.282004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:20.332153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:20.381833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994abd3-4ba5-431f-8bd8-e4979b6c8cd6 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:20.495578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c85b2-6adb-4619-b7de-985f98590d2a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:20.540600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec59c48f-9484-41f6-ac35-166dc14a0349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:20.543257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59c48f-9484-41f6-ac35-166dc14a0349 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:20.545971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59c48f-9484-41f6-ac35-166dc14a0349 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:20.547416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def8500-8554-450c-89d3-523a17754d9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:20.547638 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec59c48f-9484-41f6-ac35-166dc14a0349 admin admin] PATH_INFO: `/v3/projects/7f57144b0ee84a34bd0b68ffcb97730e/users/75774ee6cdcc4c2698731b096605da38/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:20.548242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def8500-8554-450c-89d3-523a17754d9f service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:20.548634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def8500-8554-450c-89d3-523a17754d9f service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:20.549967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def8500-8554-450c-89d3-523a17754d9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:20.552004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7def8500-8554-450c-89d3-523a17754d9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:20.556440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59c48f-9484-41f6-ac35-166dc14a0349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f57144b0ee84a34bd0b68ffcb97730e, user_id=75774ee6cdcc4c2698731b096605da38, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:20.569479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec59c48f-9484-41f6-ac35-166dc14a0349 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:20.590320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def8500-8554-450c-89d3-523a17754d9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:20.601725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def8500-8554-450c-89d3-523a17754d9f service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:20.655538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 934/1850] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:20 2022] PUT /identity/v3/projects/7f57144b0ee84a34bd0b68ffcb97730e/users/75774ee6cdcc4c2698731b096605da38/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:20.662785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c404a6c-869a-4acf-8248-d9c481df990b None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:20.664135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c404a6c-869a-4acf-8248-d9c481df990b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:20.667529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c404a6c-869a-4acf-8248-d9c481df990b None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:20.690714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 917/1851] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:20.696878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 935/1852] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:30:20 2022] GET /identity => generated 270 bytes in 40 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:30:20.697340 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12aa7f-045a-4b92-8cc4-1e816c02188f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:20.698476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12aa7f-045a-4b92-8cc4-1e816c02188f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:20.699778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12aa7f-045a-4b92-8cc4-1e816c02188f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:20.768415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978c4bb-89a6-48e3-8c96-f0ee33251977 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:20.769034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978c4bb-89a6-48e3-8c96-f0ee33251977 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:20.769483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978c4bb-89a6-48e3-8c96-f0ee33251977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:20.841588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-8978c4bb-89a6-48e3-8c96-f0ee33251977 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:20.843486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8978c4bb-89a6-48e3-8c96-f0ee33251977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:20.844944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-7a12aa7f-045a-4b92-8cc4-1e816c02188f None None] MFA Rules not processed for user `75774ee6cdcc4c2698731b096605da38`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:20.851954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7a12aa7f-045a-4b92-8cc4-1e816c02188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:21.111090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 936/1853] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:30:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:21.112994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978c4bb-89a6-48e3-8c96-f0ee33251977 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:21.115354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8978c4bb-89a6-48e3-8c96-f0ee33251977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:21.175381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 918/1854] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:21.191336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12aa7f-045a-4b92-8cc4-1e816c02188f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:21.197245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7a12aa7f-045a-4b92-8cc4-1e816c02188f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:21.288036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978c4bb-89a6-48e3-8c96-f0ee33251977 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:21.302132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:21.304782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:21.305081 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:21.305857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] PATH_INFO: `/v3/projects/f6785d2c1b524fb68c4ed995c1851497` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:21.307479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6785d2c1b524fb68c4ed995c1851497)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:21.311549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:21.338133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:21.429148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12aa7f-045a-4b92-8cc4-1e816c02188f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:21.445435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d6d4f-81b0-4120-91fc-50099e3b5525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:21.446311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d6d4f-81b0-4120-91fc-50099e3b5525 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:21.446632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d6d4f-81b0-4120-91fc-50099e3b5525 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:21.447051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d6d4f-81b0-4120-91fc-50099e3b5525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:21.467966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d6d4f-81b0-4120-91fc-50099e3b5525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:21.470435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d6d4f-81b0-4120-91fc-50099e3b5525 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:21.484084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 919/1855] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:21.523139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:21.541064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6785d2c1b524fb68c4ed995c1851497', 'request_id': 'req-33971908-8637-42a3-895f-0ef872581722'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:21.558602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6785d2c1b524fb68c4ed995c1851497', 'request_id': 'req-33971908-8637-42a3-895f-0ef872581722'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:21.574480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6785d2c1b524fb68c4ed995c1851497', 'request_id': 'req-33971908-8637-42a3-895f-0ef872581722'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:21.613641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] The token cache is being invalidate because project f6785d2c1b524fb68c4ed995c1851497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:21.614225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:21.615393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:21.621513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 937/1856] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:21 2022] DELETE /identity/v3/projects/f6785d2c1b524fb68c4ed995c1851497 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:21.629706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299320-a0a6-4496-941f-aa0b0d11de63 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:21.631342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299320-a0a6-4496-941f-aa0b0d11de63 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:21.632037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299320-a0a6-4496-941f-aa0b0d11de63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:21.673896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:21.675386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:21.677676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-26299320-a0a6-4496-941f-aa0b0d11de63 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:21.679196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-26299320-a0a6-4496-941f-aa0b0d11de63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:21.808388 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 920/1857] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:21.820211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47c6bc-49e4-41b0-a4d6-e0e3157b9578 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:21.822731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47c6bc-49e4-41b0-a4d6-e0e3157b9578 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:21.823321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47c6bc-49e4-41b0-a4d6-e0e3157b9578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:21.851465 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33971908-8637-42a3-895f-0ef872581722 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:21.859634 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:21.860737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:21.861087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:21.861345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:21.864151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:21.936142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fb47c6bc-49e4-41b0-a4d6-e0e3157b9578 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:21.940834 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fb47c6bc-49e4-41b0-a4d6-e0e3157b9578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:21.973835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 921/1858] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:21.976039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47c6bc-49e4-41b0-a4d6-e0e3157b9578 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:22.023550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47c6bc-49e4-41b0-a4d6-e0e3157b9578 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:22.047950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:22.050142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:22.050680 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c9434-138a-4ade-a714-a8222d3082d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:22.051311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c9434-138a-4ade-a714-a8222d3082d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:22.051590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c9434-138a-4ade-a714-a8222d3082d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:22.051865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c9434-138a-4ade-a714-a8222d3082d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:22.052556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c9434-138a-4ade-a714-a8222d3082d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:22.052850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c9434-138a-4ade-a714-a8222d3082d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:22.065712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c9434-138a-4ade-a714-a8222d3082d8 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:22.072210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 938/1859] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:22.087934 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:22.095754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:22.122576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 922/1860] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:22.127667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c9434-138a-4ade-a714-a8222d3082d8 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:22.147630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c9434-138a-4ade-a714-a8222d3082d8 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:22.195599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e5791-1ba3-4a44-a006-a6dd6ec6816d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:22.212315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e5791-1ba3-4a44-a006-a6dd6ec6816d admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:22.213432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e5791-1ba3-4a44-a006-a6dd6ec6816d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:22.213972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e5791-1ba3-4a44-a006-a6dd6ec6816d admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:22.214971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e5791-1ba3-4a44-a006-a6dd6ec6816d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:22.226070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e5791-1ba3-4a44-a006-a6dd6ec6816d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:22.323161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f661f7-8d04-4aa5-b296-ac424d36f7f5 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:22.337478 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88705d4-b139-4a9d-8dc6-2a62d9d94f7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:22.338000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88705d4-b139-4a9d-8dc6-2a62d9d94f7b service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:22.338281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88705d4-b139-4a9d-8dc6-2a62d9d94f7b service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:22.338735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88705d4-b139-4a9d-8dc6-2a62d9d94f7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:22.348844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d88705d4-b139-4a9d-8dc6-2a62d9d94f7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:22.350853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 923/1861] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:22 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:22.351881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e5791-1ba3-4a44-a006-a6dd6ec6816d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:22.353869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e44e5791-1ba3-4a44-a006-a6dd6ec6816d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:22.378775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88705d4-b139-4a9d-8dc6-2a62d9d94f7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:22.381103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88705d4-b139-4a9d-8dc6-2a62d9d94f7b service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:22.396256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 939/1862] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:22.397386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88705d4-b139-4a9d-8dc6-2a62d9d94f7b service nova] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:22.413547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88705d4-b139-4a9d-8dc6-2a62d9d94f7b service nova] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:22.418332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:22.498361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57833-32d3-4240-b2a4-ec8a5f02cfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:22.499113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57833-32d3-4240-b2a4-ec8a5f02cfaf admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:22.499495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57833-32d3-4240-b2a4-ec8a5f02cfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:22.499857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a57833-32d3-4240-b2a4-ec8a5f02cfaf admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:22.500530 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57833-32d3-4240-b2a4-ec8a5f02cfaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:22.503292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a57833-32d3-4240-b2a4-ec8a5f02cfaf admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:22.563951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 940/1863] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:22 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:22.568750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57833-32d3-4240-b2a4-ec8a5f02cfaf admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:22.571476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e5791-1ba3-4a44-a006-a6dd6ec6816d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:22.586763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:22.587836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:22.588122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:22.588395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] PATH_INFO: `/v3/projects/6b126e1169ce4b709c785dc40314b203` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:22.590175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b126e1169ce4b709c785dc40314b203)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:22.596876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:22.617755 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a57833-32d3-4240-b2a4-ec8a5f02cfaf admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:22.639832 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee8c7d-2c75-4088-ac27-f527608b6447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:22.641628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee8c7d-2c75-4088-ac27-f527608b6447 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:22.642164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee8c7d-2c75-4088-ac27-f527608b6447 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:22.644826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee8c7d-2c75-4088-ac27-f527608b6447 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:22.648760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee8c7d-2c75-4088-ac27-f527608b6447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:22.653793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee8c7d-2c75-4088-ac27-f527608b6447 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:22.675886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:22.686565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 941/1864] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:22.688433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee8c7d-2c75-4088-ac27-f527608b6447 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:22.724242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b126e1169ce4b709c785dc40314b203', 'request_id': 'req-96de16c5-74cf-4634-afdf-8e92e2488a4d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:22.730207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee8c7d-2c75-4088-ac27-f527608b6447 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:22.743374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b126e1169ce4b709c785dc40314b203', 'request_id': 'req-96de16c5-74cf-4634-afdf-8e92e2488a4d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:22.768432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b126e1169ce4b709c785dc40314b203', 'request_id': 'req-96de16c5-74cf-4634-afdf-8e92e2488a4d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:22.821988 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] The token cache is being invalidate because project 6b126e1169ce4b709c785dc40314b203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:22.822800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:22.824113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-96de16c5-74cf-4634-afdf-8e92e2488a4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:22.827360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 924/1865] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:22 2022] DELETE /identity/v3/projects/6b126e1169ce4b709c785dc40314b203 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:22.833893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d1618-3bab-4711-9dc2-3c27c7d91627 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:22.834475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d1618-3bab-4711-9dc2-3c27c7d91627 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:22.835704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d1618-3bab-4711-9dc2-3c27c7d91627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:22.923859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b24d1618-3bab-4711-9dc2-3c27c7d91627 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:22.925372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b24d1618-3bab-4711-9dc2-3c27c7d91627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:22.930762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960c16e-6183-4ad0-a1ab-bcf2071ac114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:22.931430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960c16e-6183-4ad0-a1ab-bcf2071ac114 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:22.931666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960c16e-6183-4ad0-a1ab-bcf2071ac114 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:22.932126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960c16e-6183-4ad0-a1ab-bcf2071ac114 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:22.933077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960c16e-6183-4ad0-a1ab-bcf2071ac114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:22.934710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960c16e-6183-4ad0-a1ab-bcf2071ac114 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:22.977007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 942/1866] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:22.977206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 925/1867] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:22.980608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960c16e-6183-4ad0-a1ab-bcf2071ac114 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:22.986094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde243c-66bb-46ac-b9a5-4b16a79ead64 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:22.986385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde243c-66bb-46ac-b9a5-4b16a79ead64 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:22.986676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde243c-66bb-46ac-b9a5-4b16a79ead64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:22.988156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8960c16e-6183-4ad0-a1ab-bcf2071ac114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:23.043820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960c16e-6183-4ad0-a1ab-bcf2071ac114 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:23.045891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-9fde243c-66bb-46ac-b9a5-4b16a79ead64 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:23.049731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9fde243c-66bb-46ac-b9a5-4b16a79ead64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:23.060117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73dd55-0a36-4717-ad11-86518d7c7f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:23.060994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73dd55-0a36-4717-ad11-86518d7c7f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.061278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73dd55-0a36-4717-ad11-86518d7c7f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.061561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73dd55-0a36-4717-ad11-86518d7c7f05 admin admin] PATH_INFO: `/v3/projects/d2616620915a49f3876922957c7c1364/users/22fb5969f768467f9f691c15269abf62/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.071486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 926/1868] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:23.076261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde243c-66bb-46ac-b9a5-4b16a79ead64 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:23.091354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73dd55-0a36-4717-ad11-86518d7c7f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2616620915a49f3876922957c7c1364, user_id=22fb5969f768467f9f691c15269abf62, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:23.092712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde243c-66bb-46ac-b9a5-4b16a79ead64 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:23.101196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73dd55-0a36-4717-ad11-86518d7c7f05 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:23.101894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2c25d-49dd-49cc-9f16-ddff7b84ac4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:23.103186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2c25d-49dd-49cc-9f16-ddff7b84ac4c admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.103687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2c25d-49dd-49cc-9f16-ddff7b84ac4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.105139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d2c25d-49dd-49cc-9f16-ddff7b84ac4c admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.106167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2c25d-49dd-49cc-9f16-ddff7b84ac4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:23.106781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2c25d-49dd-49cc-9f16-ddff7b84ac4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:23.109024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d2c25d-49dd-49cc-9f16-ddff7b84ac4c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:23.132842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 943/1869] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:22 2022] PUT /identity/v3/projects/d2616620915a49f3876922957c7c1364/users/22fb5969f768467f9f691c15269abf62/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:23.144242 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de043e07-9452-4d8d-bf5c-88c0f3a0e2b5 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.144774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de043e07-9452-4d8d-bf5c-88c0f3a0e2b5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.145186 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de043e07-9452-4d8d-bf5c-88c0f3a0e2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.156546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 927/1870] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:23.162845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2c25d-49dd-49cc-9f16-ddff7b84ac4c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:23.187591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d2c25d-49dd-49cc-9f16-ddff7b84ac4c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:23.210570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-de043e07-9452-4d8d-bf5c-88c0f3a0e2b5 None None] MFA Rules not processed for user `22fb5969f768467f9f691c15269abf62`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:23.212342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-de043e07-9452-4d8d-bf5c-88c0f3a0e2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:23.289078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:23.319546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 944/1871] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:23.320672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de043e07-9452-4d8d-bf5c-88c0f3a0e2b5 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:23.322019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-de043e07-9452-4d8d-bf5c-88c0f3a0e2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:23.350888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162072c-ee7b-418b-9576-2b0d823a4d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:23.351758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162072c-ee7b-418b-9576-2b0d823a4d45 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.352160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162072c-ee7b-418b-9576-2b0d823a4d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.352432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162072c-ee7b-418b-9576-2b0d823a4d45 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.353333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162072c-ee7b-418b-9576-2b0d823a4d45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:23.355910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162072c-ee7b-418b-9576-2b0d823a4d45 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:23.407396 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 928/1872] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:23 2022] POST /identity/v3/projects => generated 381 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:23.409082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162072c-ee7b-418b-9576-2b0d823a4d45 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:23.452482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de043e07-9452-4d8d-bf5c-88c0f3a0e2b5 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:23.516936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162072c-ee7b-418b-9576-2b0d823a4d45 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:23.524768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc099ba0-f985-452a-b59a-a9b9f3d4dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:23.526023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc099ba0-f985-452a-b59a-a9b9f3d4dfcf admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.526768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc099ba0-f985-452a-b59a-a9b9f3d4dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.527359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc099ba0-f985-452a-b59a-a9b9f3d4dfcf admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.528451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc099ba0-f985-452a-b59a-a9b9f3d4dfcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:23.532050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc099ba0-f985-452a-b59a-a9b9f3d4dfcf admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:23.549495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:23.550245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.550624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.550996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] PATH_INFO: `/v3/projects/01307ac8aa3b4b64a54b2c4999dc4966` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.553904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01307ac8aa3b4b64a54b2c4999dc4966)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:23.556746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:23.610305 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 929/1873] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:23 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:23.610305 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc099ba0-f985-452a-b59a-a9b9f3d4dfcf admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:23.612009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:23.654095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01307ac8aa3b4b64a54b2c4999dc4966', 'request_id': 'req-e73490b7-99b7-428f-8f5c-776aee5c0186'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:23.670415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01307ac8aa3b4b64a54b2c4999dc4966', 'request_id': 'req-e73490b7-99b7-428f-8f5c-776aee5c0186'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:23.683096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01307ac8aa3b4b64a54b2c4999dc4966', 'request_id': 'req-e73490b7-99b7-428f-8f5c-776aee5c0186'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:23.720375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] The token cache is being invalidate because project 01307ac8aa3b4b64a54b2c4999dc4966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:23.720873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:23.721800 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:23.724906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 945/1874] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:23 2022] DELETE /identity/v3/projects/01307ac8aa3b4b64a54b2c4999dc4966 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:23.730365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:23.732126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:23.763794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc099ba0-f985-452a-b59a-a9b9f3d4dfcf admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:23.775092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:23.851653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73490b7-99b7-428f-8f5c-776aee5c0186 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:23.865555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe0401-b33d-46df-bb53-619f8a0450e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:23.866480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe0401-b33d-46df-bb53-619f8a0450e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.866950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe0401-b33d-46df-bb53-619f8a0450e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.867604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe0401-b33d-46df-bb53-619f8a0450e8 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.868677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe0401-b33d-46df-bb53-619f8a0450e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:23.871410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe0401-b33d-46df-bb53-619f8a0450e8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:23.883249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac8f40-ac3d-45fd-96cd-19b5e5aa93c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:23.884042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac8f40-ac3d-45fd-96cd-19b5e5aa93c1 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.884722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac8f40-ac3d-45fd-96cd-19b5e5aa93c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.885347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac8f40-ac3d-45fd-96cd-19b5e5aa93c1 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.886326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac8f40-ac3d-45fd-96cd-19b5e5aa93c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:23.888398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac8f40-ac3d-45fd-96cd-19b5e5aa93c1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:23.926461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 946/1875] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:23 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:23.927911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a9483-beee-424d-9245-699560a88239 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.928323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a9483-beee-424d-9245-699560a88239 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.928958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a9483-beee-424d-9245-699560a88239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.929317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 930/1876] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:23 2022] POST /identity/v3/users => generated 291 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:23.930315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac8f40-ac3d-45fd-96cd-19b5e5aa93c1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:23.951654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac8f40-ac3d-45fd-96cd-19b5e5aa93c1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:23.962436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-363a9483-beee-424d-9245-699560a88239 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:23.963661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-363a9483-beee-424d-9245-699560a88239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:23.966155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da623c7d-b699-42b6-afb6-3e6c8fb51342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:23.966785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da623c7d-b699-42b6-afb6-3e6c8fb51342 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.967066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da623c7d-b699-42b6-afb6-3e6c8fb51342 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.967335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da623c7d-b699-42b6-afb6-3e6c8fb51342 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:23.968574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da623c7d-b699-42b6-afb6-3e6c8fb51342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:23.970330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da623c7d-b699-42b6-afb6-3e6c8fb51342 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:23.979179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 947/1877] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:23.980355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a9483-beee-424d-9245-699560a88239 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:23.982007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-363a9483-beee-424d-9245-699560a88239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:23.993583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 931/1878] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:23.993583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584457f1-8c44-4358-9b5e-fdeca80fb917 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:23.993583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584457f1-8c44-4358-9b5e-fdeca80fb917 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:23.997607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584457f1-8c44-4358-9b5e-fdeca80fb917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.002141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a9483-beee-424d-9245-699560a88239 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.007801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daae1b-47c8-46c0-9e66-8d96bfc79a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.008653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daae1b-47c8-46c0-9e66-8d96bfc79a6a admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.009175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daae1b-47c8-46c0-9e66-8d96bfc79a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.009477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daae1b-47c8-46c0-9e66-8d96bfc79a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.010710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daae1b-47c8-46c0-9e66-8d96bfc79a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.012851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daae1b-47c8-46c0-9e66-8d96bfc79a6a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.023315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-584457f1-8c44-4358-9b5e-fdeca80fb917 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:24.024559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-584457f1-8c44-4358-9b5e-fdeca80fb917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:24.028529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 948/1879] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:24.033328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daae1b-47c8-46c0-9e66-8d96bfc79a6a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.039766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 932/1880] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:24.044165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584457f1-8c44-4358-9b5e-fdeca80fb917 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.047111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daae1b-47c8-46c0-9e66-8d96bfc79a6a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.055870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287769a-39e9-423e-b890-6e457eae8059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.056920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287769a-39e9-423e-b890-6e457eae8059 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.057215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287769a-39e9-423e-b890-6e457eae8059 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.057573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287769a-39e9-423e-b890-6e457eae8059 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.058305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287769a-39e9-423e-b890-6e457eae8059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.059363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584457f1-8c44-4358-9b5e-fdeca80fb917 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.060091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287769a-39e9-423e-b890-6e457eae8059 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.068637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73680c1a-ab17-4505-b895-b28f0fa823be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.069262 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73680c1a-ab17-4505-b895-b28f0fa823be admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.069533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73680c1a-ab17-4505-b895-b28f0fa823be admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.069809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73680c1a-ab17-4505-b895-b28f0fa823be admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.072121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 949/1881] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:24.073024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287769a-39e9-423e-b890-6e457eae8059 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.075666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73680c1a-ab17-4505-b895-b28f0fa823be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.079886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73680c1a-ab17-4505-b895-b28f0fa823be admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.091198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287769a-39e9-423e-b890-6e457eae8059 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.091308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc48db-1987-48bb-9388-0fc21864ad50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.092076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc48db-1987-48bb-9388-0fc21864ad50 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.092283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc48db-1987-48bb-9388-0fc21864ad50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.092639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc48db-1987-48bb-9388-0fc21864ad50 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.093365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc48db-1987-48bb-9388-0fc21864ad50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:24.094833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc48db-1987-48bb-9388-0fc21864ad50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.097385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc48db-1987-48bb-9388-0fc21864ad50 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.104078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 933/1882] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:24.107416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73680c1a-ab17-4505-b895-b28f0fa823be admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.118664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 950/1883] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:24.120048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc48db-1987-48bb-9388-0fc21864ad50 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.122489 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73680c1a-ab17-4505-b895-b28f0fa823be admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.128335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd04de7-926f-4d57-9e11-cbe9ddbe1700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.129049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd04de7-926f-4d57-9e11-cbe9ddbe1700 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.129390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd04de7-926f-4d57-9e11-cbe9ddbe1700 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.130901 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd04de7-926f-4d57-9e11-cbe9ddbe1700 admin admin] PATH_INFO: `/v3/projects/f93cb372af3541819d6da5c3614ff81c/users/00425f7cc6a643aba2c5dbc21a42b446/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.136588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc48db-1987-48bb-9388-0fc21864ad50 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.146196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d8caf-fbe1-485a-921c-2ba14b2fce23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.147381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d8caf-fbe1-485a-921c-2ba14b2fce23 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.148407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d8caf-fbe1-485a-921c-2ba14b2fce23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.149121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d8caf-fbe1-485a-921c-2ba14b2fce23 admin admin] PATH_INFO: `/v3/projects/62580d62435b452baf5d367ef3615a11/users/9d25d474ab39400da0c35a904799fdce/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.151377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd04de7-926f-4d57-9e11-cbe9ddbe1700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93cb372af3541819d6da5c3614ff81c, user_id=00425f7cc6a643aba2c5dbc21a42b446, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.157152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd04de7-926f-4d57-9e11-cbe9ddbe1700 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.187566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 934/1884] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:24 2022] PUT /identity/v3/projects/f93cb372af3541819d6da5c3614ff81c/users/00425f7cc6a643aba2c5dbc21a42b446/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:24.187566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd04de7-926f-4d57-9e11-cbe9ddbe1700 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.190255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d8caf-fbe1-485a-921c-2ba14b2fce23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62580d62435b452baf5d367ef3615a11, user_id=9d25d474ab39400da0c35a904799fdce, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.193934 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d8caf-fbe1-485a-921c-2ba14b2fce23 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.216807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 951/1885] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:24 2022] PUT /identity/v3/projects/62580d62435b452baf5d367ef3615a11/users/9d25d474ab39400da0c35a904799fdce/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:24.223944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1ee1e-066d-4cf8-abb1-1588a3252b48 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.224385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1ee1e-066d-4cf8-abb1-1588a3252b48 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.224955 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f1ee1e-066d-4cf8-abb1-1588a3252b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.259374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:24.277995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd04de7-926f-4d57-9e11-cbe9ddbe1700 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.292282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-20f1ee1e-066d-4cf8-abb1-1588a3252b48 None None] MFA Rules not processed for user `00425f7cc6a643aba2c5dbc21a42b446`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:24.293690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-20f1ee1e-066d-4cf8-abb1-1588a3252b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:24.376761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda45112-713c-450d-a3e3-18fb153cba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.376761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda45112-713c-450d-a3e3-18fb153cba51 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.376761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda45112-713c-450d-a3e3-18fb153cba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.376761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda45112-713c-450d-a3e3-18fb153cba51 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.377348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda45112-713c-450d-a3e3-18fb153cba51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.379127 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda45112-713c-450d-a3e3-18fb153cba51 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.402586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 952/1886] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:24.405708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5aa83-caba-4e62-b00a-03b7c80fb8fd None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.405708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5aa83-caba-4e62-b00a-03b7c80fb8fd None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.405708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5aa83-caba-4e62-b00a-03b7c80fb8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.427297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 935/1887] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:24 2022] POST /identity/v3/projects => generated 387 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:24.429419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda45112-713c-450d-a3e3-18fb153cba51 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.451952 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-75c5aa83-caba-4e62-b00a-03b7c80fb8fd None None] MFA Rules not processed for user `9d25d474ab39400da0c35a904799fdce`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:24.453329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-75c5aa83-caba-4e62-b00a-03b7c80fb8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:24.508551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:24.539413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda45112-713c-450d-a3e3-18fb153cba51 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.545179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6253be8-8563-4320-b19a-8cc457200cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.546610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6253be8-8563-4320-b19a-8cc457200cfe admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.547269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6253be8-8563-4320-b19a-8cc457200cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.547810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6253be8-8563-4320-b19a-8cc457200cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.548577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6253be8-8563-4320-b19a-8cc457200cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.551008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6253be8-8563-4320-b19a-8cc457200cfe admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.590518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 953/1888] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:24.591057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5aa83-caba-4e62-b00a-03b7c80fb8fd None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.594995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 936/1889] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:24 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:24.595907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6253be8-8563-4320-b19a-8cc457200cfe admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.605697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5aa83-caba-4e62-b00a-03b7c80fb8fd None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.683208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81e7e5-04e3-4a01-992d-85925e5a4475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.683920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e7e5-04e3-4a01-992d-85925e5a4475 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.684443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e7e5-04e3-4a01-992d-85925e5a4475 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.684863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81e7e5-04e3-4a01-992d-85925e5a4475 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.685505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81e7e5-04e3-4a01-992d-85925e5a4475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.688431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81e7e5-04e3-4a01-992d-85925e5a4475 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.714578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6253be8-8563-4320-b19a-8cc457200cfe admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.725720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0ac99-23cd-48a8-a9a9-9571ab590f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.726584 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0ac99-23cd-48a8-a9a9-9571ab590f24 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.726896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0ac99-23cd-48a8-a9a9-9571ab590f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.727215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0ac99-23cd-48a8-a9a9-9571ab590f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.728375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0ac99-23cd-48a8-a9a9-9571ab590f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.730973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0ac99-23cd-48a8-a9a9-9571ab590f24 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.735119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 954/1890] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:24 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:24.736123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81e7e5-04e3-4a01-992d-85925e5a4475 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.749870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:24.753032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81e7e5-04e3-4a01-992d-85925e5a4475 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.760153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f5d95-0625-4ee3-8e3d-25f4274308c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.761498 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f5d95-0625-4ee3-8e3d-25f4274308c5 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.762505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f5d95-0625-4ee3-8e3d-25f4274308c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.762788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f5d95-0625-4ee3-8e3d-25f4274308c5 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.763652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f5d95-0625-4ee3-8e3d-25f4274308c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.766457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f5d95-0625-4ee3-8e3d-25f4274308c5 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.784759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 937/1891] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:24 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:24.786469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0ac99-23cd-48a8-a9a9-9571ab590f24 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.797774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 955/1892] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:24 2022] POST /identity/v3/users => generated 295 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:24.799183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f5d95-0625-4ee3-8e3d-25f4274308c5 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.820653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f5d95-0625-4ee3-8e3d-25f4274308c5 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.871308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0ac99-23cd-48a8-a9a9-9571ab590f24 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.877635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5947c1-a2c2-4a47-8afc-51ef0f16aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.878330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5947c1-a2c2-4a47-8afc-51ef0f16aaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.878652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5947c1-a2c2-4a47-8afc-51ef0f16aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.880074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5947c1-a2c2-4a47-8afc-51ef0f16aaf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.880851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5947c1-a2c2-4a47-8afc-51ef0f16aaf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.883357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5947c1-a2c2-4a47-8afc-51ef0f16aaf3 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.892369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f36841-ee3a-43a8-bfad-4c3bd1efe150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.893262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f36841-ee3a-43a8-bfad-4c3bd1efe150 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.893444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f36841-ee3a-43a8-bfad-4c3bd1efe150 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.893665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f36841-ee3a-43a8-bfad-4c3bd1efe150 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.894306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f36841-ee3a-43a8-bfad-4c3bd1efe150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.897069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 938/1893] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:24.897069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5947c1-a2c2-4a47-8afc-51ef0f16aaf3 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.897281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f36841-ee3a-43a8-bfad-4c3bd1efe150 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.912434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5947c1-a2c2-4a47-8afc-51ef0f16aaf3 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.918011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073ede4-7736-4147-9b5a-e669af6f40e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.918630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073ede4-7736-4147-9b5a-e669af6f40e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.918913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073ede4-7736-4147-9b5a-e669af6f40e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.919227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073ede4-7736-4147-9b5a-e669af6f40e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.919832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073ede4-7736-4147-9b5a-e669af6f40e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.922341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073ede4-7736-4147-9b5a-e669af6f40e4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.923191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 956/1894] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:24 2022] POST /identity/v3/users => generated 297 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:24.924049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f36841-ee3a-43a8-bfad-4c3bd1efe150 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.933876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f36841-ee3a-43a8-bfad-4c3bd1efe150 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.938709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed25e9-ce31-453e-8b92-d815275809e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.939302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed25e9-ce31-453e-8b92-d815275809e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.939580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed25e9-ce31-453e-8b92-d815275809e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.939849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed25e9-ce31-453e-8b92-d815275809e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.940516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed25e9-ce31-453e-8b92-d815275809e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.942129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed25e9-ce31-453e-8b92-d815275809e7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.948409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 939/1895] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:24.949156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073ede4-7736-4147-9b5a-e669af6f40e4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.957386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 957/1896] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:24.958500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed25e9-ce31-453e-8b92-d815275809e7 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:24.962015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073ede4-7736-4147-9b5a-e669af6f40e4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.970322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8018-4e84-4e4d-9d53-ae16ed04a137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.970989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8018-4e84-4e4d-9d53-ae16ed04a137 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.971327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed25e9-ce31-453e-8b92-d815275809e7 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:24.971415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8018-4e84-4e4d-9d53-ae16ed04a137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.971639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b8018-4e84-4e4d-9d53-ae16ed04a137 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.972777 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b8018-4e84-4e4d-9d53-ae16ed04a137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:24.974811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b8018-4e84-4e4d-9d53-ae16ed04a137 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:24.986753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc8c18-05c6-472c-b628-88a796e6c565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:24.987965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc8c18-05c6-472c-b628-88a796e6c565 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:24.988461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc8c18-05c6-472c-b628-88a796e6c565 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:24.989008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc8c18-05c6-472c-b628-88a796e6c565 admin admin] PATH_INFO: `/v3/projects/bb4dcaf5d5cd47f48ea5271b437ddf41/users/8e4cb151054942dbae158d8f504188c9/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:24.992413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 940/1897] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:24.993682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8018-4e84-4e4d-9d53-ae16ed04a137 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.006473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b8018-4e84-4e4d-9d53-ae16ed04a137 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:25.012045 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac302cce-94c0-40ba-ab18-f08c3251b4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:25.012668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac302cce-94c0-40ba-ab18-f08c3251b4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.012950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac302cce-94c0-40ba-ab18-f08c3251b4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.013059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc8c18-05c6-472c-b628-88a796e6c565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4dcaf5d5cd47f48ea5271b437ddf41, user_id=8e4cb151054942dbae158d8f504188c9, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.013228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac302cce-94c0-40ba-ab18-f08c3251b4d4 admin admin] PATH_INFO: `/v3/projects/8241847026654a62a22240aed131da1b/users/2ed9932428b74c25a22ab4eb39d0db60/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.018227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc8c18-05c6-472c-b628-88a796e6c565 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.021551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:25.037784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac302cce-94c0-40ba-ab18-f08c3251b4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8241847026654a62a22240aed131da1b, user_id=2ed9932428b74c25a22ab4eb39d0db60, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.040082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 958/1898] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:24 2022] PUT /identity/v3/projects/bb4dcaf5d5cd47f48ea5271b437ddf41/users/8e4cb151054942dbae158d8f504188c9/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:25.041006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac302cce-94c0-40ba-ab18-f08c3251b4d4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.041408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc8c18-05c6-472c-b628-88a796e6c565 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.068138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 941/1899] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:24 2022] PUT /identity/v3/projects/8241847026654a62a22240aed131da1b/users/2ed9932428b74c25a22ab4eb39d0db60/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:25.068138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac302cce-94c0-40ba-ab18-f08c3251b4d4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.121076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc8c18-05c6-472c-b628-88a796e6c565 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:25.144413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac302cce-94c0-40ba-ab18-f08c3251b4d4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:25.153031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9f859-efbe-4b63-bae9-bbe709d9d59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:25.153868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9f859-efbe-4b63-bae9-bbe709d9d59e admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.154150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9f859-efbe-4b63-bae9-bbe709d9d59e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.154557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9f859-efbe-4b63-bae9-bbe709d9d59e admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.156155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9f859-efbe-4b63-bae9-bbe709d9d59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.159289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9f859-efbe-4b63-bae9-bbe709d9d59e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.178584 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 942/1900] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:25.181964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a2222-2628-40a0-8105-65ba2daf4234 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.182252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a2222-2628-40a0-8105-65ba2daf4234 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.182526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a2222-2628-40a0-8105-65ba2daf4234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.194257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fc103-c4fb-4276-b192-b93aee0d0bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:25.194935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fc103-c4fb-4276-b192-b93aee0d0bae admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.195209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fc103-c4fb-4276-b192-b93aee0d0bae admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.195481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fc103-c4fb-4276-b192-b93aee0d0bae admin admin] PATH_INFO: `/v3/projects/bff421afdfb6422c8a60c80cc6591116/users/8bfc3658c16245f6ac3d06d25f9cfc31/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.222371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fc103-c4fb-4276-b192-b93aee0d0bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff421afdfb6422c8a60c80cc6591116, user_id=8bfc3658c16245f6ac3d06d25f9cfc31, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.225562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fc103-c4fb-4276-b192-b93aee0d0bae admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.239754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-f83a2222-2628-40a0-8105-65ba2daf4234 None None] MFA Rules not processed for user `2ed9932428b74c25a22ab4eb39d0db60`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:25.240993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f83a2222-2628-40a0-8105-65ba2daf4234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:25.246717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 959/1901] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:25 2022] PUT /identity/v3/projects/bff421afdfb6422c8a60c80cc6591116/users/8bfc3658c16245f6ac3d06d25f9cfc31/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:25.248028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fc103-c4fb-4276-b192-b93aee0d0bae admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.307323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:25.360027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fc103-c4fb-4276-b192-b93aee0d0bae admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:25.365725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14251cf3-136b-4bd1-bbaf-6f0c81fc80c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:25.366411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14251cf3-136b-4bd1-bbaf-6f0c81fc80c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.366696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14251cf3-136b-4bd1-bbaf-6f0c81fc80c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.366970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14251cf3-136b-4bd1-bbaf-6f0c81fc80c6 admin admin] PATH_INFO: `/v3/projects/bb4dcaf5d5cd47f48ea5271b437ddf41/users/8e4cb151054942dbae158d8f504188c9/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.369169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14251cf3-136b-4bd1-bbaf-6f0c81fc80c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4dcaf5d5cd47f48ea5271b437ddf41, user_id=8e4cb151054942dbae158d8f504188c9, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.372267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14251cf3-136b-4bd1-bbaf-6f0c81fc80c6 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.386386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 960/1902] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:25 2022] PUT /identity/v3/projects/bb4dcaf5d5cd47f48ea5271b437ddf41/users/8e4cb151054942dbae158d8f504188c9/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:25.395427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14251cf3-136b-4bd1-bbaf-6f0c81fc80c6 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.423463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 943/1903] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:25.425794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c19b-4b7d-4cd4-a614-93a9b10d36fb None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.425794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c19b-4b7d-4cd4-a614-93a9b10d36fb None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.425794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624c19b-4b7d-4cd4-a614-93a9b10d36fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.441826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:25.484076 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e624c19b-4b7d-4cd4-a614-93a9b10d36fb None None] MFA Rules not processed for user `8e4cb151054942dbae158d8f504188c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:25.485694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e624c19b-4b7d-4cd4-a614-93a9b10d36fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:25.506620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14251cf3-136b-4bd1-bbaf-6f0c81fc80c6 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:25.511569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771bf6d-55d7-4537-9fd8-812dc1cd2c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:25.512213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771bf6d-55d7-4537-9fd8-812dc1cd2c8a admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.512527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771bf6d-55d7-4537-9fd8-812dc1cd2c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.512804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771bf6d-55d7-4537-9fd8-812dc1cd2c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.513510 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771bf6d-55d7-4537-9fd8-812dc1cd2c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.515774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771bf6d-55d7-4537-9fd8-812dc1cd2c8a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.532815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 961/1904] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:25.534426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771bf6d-55d7-4537-9fd8-812dc1cd2c8a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.549306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771bf6d-55d7-4537-9fd8-812dc1cd2c8a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:25.554188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531e0e5-ab8f-4d44-b47c-c3558c2e0633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:25.554797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531e0e5-ab8f-4d44-b47c-c3558c2e0633 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.555077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531e0e5-ab8f-4d44-b47c-c3558c2e0633 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.555355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531e0e5-ab8f-4d44-b47c-c3558c2e0633 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.555994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531e0e5-ab8f-4d44-b47c-c3558c2e0633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.557991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531e0e5-ab8f-4d44-b47c-c3558c2e0633 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.620412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 944/1905] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:25.620412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624c19b-4b7d-4cd4-a614-93a9b10d36fb None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.636357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624c19b-4b7d-4cd4-a614-93a9b10d36fb None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:25.642369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9eb39-bde6-4ed8-86f5-69933667a3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:25.643253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9eb39-bde6-4ed8-86f5-69933667a3e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.643537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9eb39-bde6-4ed8-86f5-69933667a3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.644033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9eb39-bde6-4ed8-86f5-69933667a3e1 admin admin] PATH_INFO: `/v3/projects/bff421afdfb6422c8a60c80cc6591116/users/8bfc3658c16245f6ac3d06d25f9cfc31/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.646341 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 962/1906] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:25 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:25.647455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531e0e5-ab8f-4d44-b47c-c3558c2e0633 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.648125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9eb39-bde6-4ed8-86f5-69933667a3e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff421afdfb6422c8a60c80cc6591116, user_id=8bfc3658c16245f6ac3d06d25f9cfc31, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.651507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2531e0e5-ab8f-4d44-b47c-c3558c2e0633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:25.652909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9eb39-bde6-4ed8-86f5-69933667a3e1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.676603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 945/1907] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:25 2022] PUT /identity/v3/projects/bff421afdfb6422c8a60c80cc6591116/users/8bfc3658c16245f6ac3d06d25f9cfc31/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:25.676603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9eb39-bde6-4ed8-86f5-69933667a3e1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.676603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-50b9eb39-bde6-4ed8-86f5-69933667a3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:25.766157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531e0e5-ab8f-4d44-b47c-c3558c2e0633 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:25.769921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9eb39-bde6-4ed8-86f5-69933667a3e1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:25.775868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:25.776653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.777030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.777638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] PATH_INFO: `/v3/users/ccc1163b3ca542ae81af1a9e069987d3` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.779063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc1163b3ca542ae81af1a9e069987d3)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.782005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.837200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:25.847081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc1163b3ca542ae81af1a9e069987d3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:25.848664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc1163b3ca542ae81af1a9e069987d3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:25.856372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3995db7-423a-4409-b61a-99728733c1c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:25.857300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995db7-423a-4409-b61a-99728733c1c2 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.857585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995db7-423a-4409-b61a-99728733c1c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.857880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995db7-423a-4409-b61a-99728733c1c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:25.868659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc1163b3ca542ae81af1a9e069987d3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:25.877224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc1163b3ca542ae81af1a9e069987d3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:25.886771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc1163b3ca542ae81af1a9e069987d3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:25.898831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 946/1908] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:25 2022] DELETE /identity/v3/users/ccc1163b3ca542ae81af1a9e069987d3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:25.900883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:25.903680 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:25.943392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3995db7-423a-4409-b61a-99728733c1c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:25.945505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3995db7-423a-4409-b61a-99728733c1c2 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:25.951776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:25.963179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 963/1909] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:25.965077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e203990-8312-49a5-b257-dafcb4ed8e06 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:25.965525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e203990-8312-49a5-b257-dafcb4ed8e06 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:25.965828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e203990-8312-49a5-b257-dafcb4ed8e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.031875 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cd22e-20f3-4f48-876c-b21fbc7b54f6 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:26.037244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb5508-de54-4c37-8cd4-8dcc626939a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:26.037987 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb5508-de54-4c37-8cd4-8dcc626939a5 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.038364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb5508-de54-4c37-8cd4-8dcc626939a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.038888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb5508-de54-4c37-8cd4-8dcc626939a5 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.039547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb5508-de54-4c37-8cd4-8dcc626939a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:26.041792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb5508-de54-4c37-8cd4-8dcc626939a5 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:26.053011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6e203990-8312-49a5-b257-dafcb4ed8e06 None None] MFA Rules not processed for user `8bfc3658c16245f6ac3d06d25f9cfc31`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:26.054592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6e203990-8312-49a5-b257-dafcb4ed8e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:26.078617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 947/1910] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:25 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:26.079831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1eac3-2ec5-4e21-a263-a0571fe4f683 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.081541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1eac3-2ec5-4e21-a263-a0571fe4f683 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.081822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1eac3-2ec5-4e21-a263-a0571fe4f683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.109516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-afc1eac3-2ec5-4e21-a263-a0571fe4f683 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:26.111228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-afc1eac3-2ec5-4e21-a263-a0571fe4f683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:26.121098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 948/1911] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:26.123183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1eac3-2ec5-4e21-a263-a0571fe4f683 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:26.125866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-afc1eac3-2ec5-4e21-a263-a0571fe4f683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:26.195655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 964/1912] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:26.195895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e203990-8312-49a5-b257-dafcb4ed8e06 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:26.210484 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e203990-8312-49a5-b257-dafcb4ed8e06 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:26.214560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1eac3-2ec5-4e21-a263-a0571fe4f683 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:26.216174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286d157-0155-42b6-bb94-7aca128df05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:26.216983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286d157-0155-42b6-bb94-7aca128df05f admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.217338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286d157-0155-42b6-bb94-7aca128df05f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.217707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0286d157-0155-42b6-bb94-7aca128df05f admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.219333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939321a-4f59-4fa5-a5ca-9ebb98cd547b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:26.219499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0286d157-0155-42b6-bb94-7aca128df05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:26.220045 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939321a-4f59-4fa5-a5ca-9ebb98cd547b service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.220382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939321a-4f59-4fa5-a5ca-9ebb98cd547b service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.220693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939321a-4f59-4fa5-a5ca-9ebb98cd547b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.221073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0286d157-0155-42b6-bb94-7aca128df05f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:26.222800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a939321a-4f59-4fa5-a5ca-9ebb98cd547b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:26.235625 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 965/1913] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:26.237552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286d157-0155-42b6-bb94-7aca128df05f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:26.242971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0286d157-0155-42b6-bb94-7aca128df05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:26.319953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0286d157-0155-42b6-bb94-7aca128df05f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:26.324524 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e5e7d-0e38-4478-b4ad-3a5c3d3db62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:26.325167 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e5e7d-0e38-4478-b4ad-3a5c3d3db62d service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.325445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e5e7d-0e38-4478-b4ad-3a5c3d3db62d service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.325718 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e5e7d-0e38-4478-b4ad-3a5c3d3db62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.337984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e5e7d-0e38-4478-b4ad-3a5c3d3db62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:26.339479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:26.339613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e5e7d-0e38-4478-b4ad-3a5c3d3db62d service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:26.349203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939321a-4f59-4fa5-a5ca-9ebb98cd547b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:26.352588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939321a-4f59-4fa5-a5ca-9ebb98cd547b service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:26.363085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 966/1914] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:26.364135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e5e7d-0e38-4478-b4ad-3a5c3d3db62d service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:26.366301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-353e5e7d-0e38-4478-b4ad-3a5c3d3db62d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:26.373781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 949/1915] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:26.379259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939321a-4f59-4fa5-a5ca-9ebb98cd547b service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:26.398024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e5e7d-0e38-4478-b4ad-3a5c3d3db62d service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:26.408762 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939321a-4f59-4fa5-a5ca-9ebb98cd547b service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:26.410225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9fe5c-7ee9-45e0-af5f-06ee72719891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:26.410849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9fe5c-7ee9-45e0-af5f-06ee72719891 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.411156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9fe5c-7ee9-45e0-af5f-06ee72719891 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.411457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9fe5c-7ee9-45e0-af5f-06ee72719891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.425230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77c3f8-4c44-49fc-b066-1a160f63c207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:26.425947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77c3f8-4c44-49fc-b066-1a160f63c207 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.426706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77c3f8-4c44-49fc-b066-1a160f63c207 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.426706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77c3f8-4c44-49fc-b066-1a160f63c207 admin admin] PATH_INFO: `/v3/projects/116f1ab7b5af42c7a59a846d848de022/users/19a227b9aa85468fbd81d4b313f7c449/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.436131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9fe5c-7ee9-45e0-af5f-06ee72719891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:26.438016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9fe5c-7ee9-45e0-af5f-06ee72719891 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:26.464981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77c3f8-4c44-49fc-b066-1a160f63c207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116f1ab7b5af42c7a59a846d848de022, user_id=19a227b9aa85468fbd81d4b313f7c449, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:26.469324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 967/1916] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:26.470845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77c3f8-4c44-49fc-b066-1a160f63c207 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:26.498056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 950/1917] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:26 2022] PUT /identity/v3/projects/116f1ab7b5af42c7a59a846d848de022/users/19a227b9aa85468fbd81d4b313f7c449/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:26.502496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9fe5c-7ee9-45e0-af5f-06ee72719891 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:26.522104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9fe5c-7ee9-45e0-af5f-06ee72719891 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:26.623173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce870d1-4349-45e6-8a81-09ee18d73ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:26.624166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce870d1-4349-45e6-8a81-09ee18d73ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.624584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce870d1-4349-45e6-8a81-09ee18d73ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.625180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce870d1-4349-45e6-8a81-09ee18d73ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.626442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce870d1-4349-45e6-8a81-09ee18d73ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:26.629324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce870d1-4349-45e6-8a81-09ee18d73ef7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:26.632900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:26.663190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 968/1918] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:26.667260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77c3f8-4c44-49fc-b066-1a160f63c207 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:26.681452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77c3f8-4c44-49fc-b066-1a160f63c207 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:26.690845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ab2d2-0b23-4c90-837a-efcb5857dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:26.691535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ab2d2-0b23-4c90-837a-efcb5857dec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.691909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ab2d2-0b23-4c90-837a-efcb5857dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.692280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ab2d2-0b23-4c90-837a-efcb5857dec2 admin admin] PATH_INFO: `/v3/projects/116f1ab7b5af42c7a59a846d848de022/users/19a227b9aa85468fbd81d4b313f7c449/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.694945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ab2d2-0b23-4c90-837a-efcb5857dec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116f1ab7b5af42c7a59a846d848de022, user_id=19a227b9aa85468fbd81d4b313f7c449, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:26.699189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ab2d2-0b23-4c90-837a-efcb5857dec2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:26.723450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 951/1919] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:26 2022] PUT /identity/v3/projects/116f1ab7b5af42c7a59a846d848de022/users/19a227b9aa85468fbd81d4b313f7c449/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:26.738327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d45e5-a3cd-484b-ba4f-aad333cd5080 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:26.741015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d45e5-a3cd-484b-ba4f-aad333cd5080 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:26.741376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d45e5-a3cd-484b-ba4f-aad333cd5080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:26.810270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-287d45e5-a3cd-484b-ba4f-aad333cd5080 None None] MFA Rules not processed for user `19a227b9aa85468fbd81d4b313f7c449`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:26.810270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-287d45e5-a3cd-484b-ba4f-aad333cd5080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:26.992821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 969/1920] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:27.033092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ab2d2-0b23-4c90-837a-efcb5857dec2 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:27.145235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ab2d2-0b23-4c90-837a-efcb5857dec2 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:27.149186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169216d-6155-480c-bc3e-047c60f4a145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:27.149806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169216d-6155-480c-bc3e-047c60f4a145 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:27.150122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169216d-6155-480c-bc3e-047c60f4a145 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:27.150483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169216d-6155-480c-bc3e-047c60f4a145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:27.162264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169216d-6155-480c-bc3e-047c60f4a145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:27.163894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169216d-6155-480c-bc3e-047c60f4a145 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:27.185226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 952/1921] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:27.324127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d45e5-a3cd-484b-ba4f-aad333cd5080 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:27.325886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-287d45e5-a3cd-484b-ba4f-aad333cd5080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:27.513533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169216d-6155-480c-bc3e-047c60f4a145 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:27.525691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d45e5-a3cd-484b-ba4f-aad333cd5080 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:27.529832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169216d-6155-480c-bc3e-047c60f4a145 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:27.532474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:27.533615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:27.534027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:27.534309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] PATH_INFO: `/v3/users/9d25d474ab39400da0c35a904799fdce` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:27.535217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:27.535668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d25d474ab39400da0c35a904799fdce)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:27.535915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:27.536193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:27.536462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] PATH_INFO: `/v3/users/2ed9932428b74c25a22ab4eb39d0db60` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:27.537810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed9932428b74c25a22ab4eb39d0db60)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:27.538512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:27.539994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:27.608123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d25d474ab39400da0c35a904799fdce'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.610835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d25d474ab39400da0c35a904799fdce'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.618364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed9932428b74c25a22ab4eb39d0db60'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.619102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed9932428b74c25a22ab4eb39d0db60'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.620582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d25d474ab39400da0c35a904799fdce'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.630415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d25d474ab39400da0c35a904799fdce'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.631322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-76c2b77a-a648-40f6-a0c3-fb5648a9e25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d25d474ab39400da0c35a904799fdce'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.633153 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed9932428b74c25a22ab4eb39d0db60'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.641038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed9932428b74c25a22ab4eb39d0db60'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.649907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b443c2d9-6aab-44d0-992e-3003a37b92e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed9932428b74c25a22ab4eb39d0db60'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:27.654950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 970/1922] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:27 2022] DELETE /identity/v3/users/9d25d474ab39400da0c35a904799fdce => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:27.660432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ce177-c825-4378-a8fa-95442dbc1a9d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:27.660878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ce177-c825-4378-a8fa-95442dbc1a9d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:27.661370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ce177-c825-4378-a8fa-95442dbc1a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:27.668909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:27.676465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 953/1923] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:27 2022] DELETE /identity/v3/users/2ed9932428b74c25a22ab4eb39d0db60 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:27.678547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28d0ad-0bf9-4c00-8607-5e0c5168a57c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:27.678724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28d0ad-0bf9-4c00-8607-5e0c5168a57c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:27.679002 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28d0ad-0bf9-4c00-8607-5e0c5168a57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:27.694479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a97ce177-c825-4378-a8fa-95442dbc1a9d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:27.695352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a97ce177-c825-4378-a8fa-95442dbc1a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:27.710828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fd28d0ad-0bf9-4c00-8607-5e0c5168a57c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:27.712135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fd28d0ad-0bf9-4c00-8607-5e0c5168a57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:27.770182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:27.788957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 971/1924] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:27.831822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 954/1925] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:27.847247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ce177-c825-4378-a8fa-95442dbc1a9d None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:27.851096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a97ce177-c825-4378-a8fa-95442dbc1a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:27.855556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28d0ad-0bf9-4c00-8607-5e0c5168a57c None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:27.859150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fd28d0ad-0bf9-4c00-8607-5e0c5168a57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:27.972741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28d0ad-0bf9-4c00-8607-5e0c5168a57c None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:27.978297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee99075-4881-4b0d-877b-3caa8eac6877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:27.979111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee99075-4881-4b0d-877b-3caa8eac6877 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:27.979563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee99075-4881-4b0d-877b-3caa8eac6877 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:27.979980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee99075-4881-4b0d-877b-3caa8eac6877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:27.991019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee99075-4881-4b0d-877b-3caa8eac6877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:27.992594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee99075-4881-4b0d-877b-3caa8eac6877 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:28.007020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 955/1926] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:28.010111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ce177-c825-4378-a8fa-95442dbc1a9d None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:28.015385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f7eaf-5555-40a1-8c90-d4ac61e4c2c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:28.016129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f7eaf-5555-40a1-8c90-d4ac61e4c2c5 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:28.016404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f7eaf-5555-40a1-8c90-d4ac61e4c2c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:28.016843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f7eaf-5555-40a1-8c90-d4ac61e4c2c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:28.028020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f7eaf-5555-40a1-8c90-d4ac61e4c2c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:28.029939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f7eaf-5555-40a1-8c90-d4ac61e4c2c5 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:28.042125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 972/1927] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:28.639272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee99075-4881-4b0d-877b-3caa8eac6877 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:28.642727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0ee99075-4881-4b0d-877b-3caa8eac6877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:28.664469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee99075-4881-4b0d-877b-3caa8eac6877 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:28.670235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:28.671848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:28.672349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:28.672725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] PATH_INFO: `/v3/projects/0b9b4ab5a7e64c6fb3d414db8c2d0b9d` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:28.674454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9b4ab5a7e64c6fb3d414db8c2d0b9d)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:28.676770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:28.794025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9b4ab5a7e64c6fb3d414db8c2d0b9d', 'request_id': 'req-91c3db0d-df47-4e18-9925-43150419c787'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:28.803035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9b4ab5a7e64c6fb3d414db8c2d0b9d', 'request_id': 'req-91c3db0d-df47-4e18-9925-43150419c787'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:28.834350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9b4ab5a7e64c6fb3d414db8c2d0b9d', 'request_id': 'req-91c3db0d-df47-4e18-9925-43150419c787'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:28.875430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f7eaf-5555-40a1-8c90-d4ac61e4c2c5 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:28.878980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-714f7eaf-5555-40a1-8c90-d4ac61e4c2c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:28.914248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] The token cache is being invalidate because project 0b9b4ab5a7e64c6fb3d414db8c2d0b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:28.916183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:28.920083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:28.933761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f7eaf-5555-40a1-8c90-d4ac61e4c2c5 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:28.933902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 956/1928] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:28 2022] DELETE /identity/v3/projects/0b9b4ab5a7e64c6fb3d414db8c2d0b9d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:28.934915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:28.942142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:29.103773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:29.104811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:29.105168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:29.106241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] PATH_INFO: `/v3/projects/8241847026654a62a22240aed131da1b` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:29.108092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8241847026654a62a22240aed131da1b)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:29.111797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3db0d-df47-4e18-9925-43150419c787 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:29.119691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:29.131067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:29.131766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:29.132004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:29.132298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] PATH_INFO: `/v3/users/75774ee6cdcc4c2698731b096605da38` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:29.138410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] RBAC: Authorizing `identity:delete_user(user_id=75774ee6cdcc4c2698731b096605da38)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:29.144282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:29.152139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:29.244298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:29.258793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8241847026654a62a22240aed131da1b', 'request_id': 'req-46cb7207-f189-47bc-96e4-68d6b4342a14'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.273533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8241847026654a62a22240aed131da1b', 'request_id': 'req-46cb7207-f189-47bc-96e4-68d6b4342a14'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.276162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75774ee6cdcc4c2698731b096605da38'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.278214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75774ee6cdcc4c2698731b096605da38'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.285136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8241847026654a62a22240aed131da1b', 'request_id': 'req-46cb7207-f189-47bc-96e4-68d6b4342a14'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.310488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75774ee6cdcc4c2698731b096605da38'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.314613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] The token cache is being invalidate because project 8241847026654a62a22240aed131da1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:29.314987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.315799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.319047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 973/1929] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:28 2022] DELETE /identity/v3/projects/8241847026654a62a22240aed131da1b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:29.320032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:29.321096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:29.344326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75774ee6cdcc4c2698731b096605da38'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.360770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75774ee6cdcc4c2698731b096605da38'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.378215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 957/1930] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:28 2022] DELETE /identity/v3/users/75774ee6cdcc4c2698731b096605da38 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:29.379091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:29.381687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:29.465871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7207-f189-47bc-96e4-68d6b4342a14 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:29.574018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e1ad0-1a84-412f-afce-e4c0ce2a884f admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:29.596428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:29.597137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:29.597418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:29.597690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] PATH_INFO: `/v3/users/19a227b9aa85468fbd81d4b313f7c449` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:29.599069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a227b9aa85468fbd81d4b313f7c449)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:29.606259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:29.620843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:29.621507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:29.621788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:29.622169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] PATH_INFO: `/v3/projects/62580d62435b452baf5d367ef3615a11` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:29.623356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62580d62435b452baf5d367ef3615a11)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:29.627034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:29.720413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a227b9aa85468fbd81d4b313f7c449'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.722914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a227b9aa85468fbd81d4b313f7c449'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.762073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62580d62435b452baf5d367ef3615a11', 'request_id': 'req-8f541ca1-24de-4b47-8a38-6511b31ca232'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.766096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a227b9aa85468fbd81d4b313f7c449'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.781374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62580d62435b452baf5d367ef3615a11', 'request_id': 'req-8f541ca1-24de-4b47-8a38-6511b31ca232'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.793793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a227b9aa85468fbd81d4b313f7c449'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.798037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62580d62435b452baf5d367ef3615a11', 'request_id': 'req-8f541ca1-24de-4b47-8a38-6511b31ca232'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.808588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0b778602-60d0-463d-8644-cac9c8d18514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a227b9aa85468fbd81d4b313f7c449'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.810553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:29.828034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 958/1931] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:29 2022] DELETE /identity/v3/users/19a227b9aa85468fbd81d4b313f7c449 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:29.830146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f8692-901e-4415-8be3-ced78997499a None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:29.832194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f8692-901e-4415-8be3-ced78997499a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:29.832536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f8692-901e-4415-8be3-ced78997499a None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:29.833115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] The token cache is being invalidate because project 62580d62435b452baf5d367ef3615a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:29.833414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.835175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 959/1932] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:30:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:30:29.835282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8f541ca1-24de-4b47-8a38-6511b31ca232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:29.838139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 974/1933] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:29 2022] DELETE /identity/v3/projects/62580d62435b452baf5d367ef3615a11 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:29.842719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f8692-901e-4415-8be3-ced78997499a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:29.845096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-833f8692-901e-4415-8be3-ced78997499a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:29.849879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14ff78-79d3-4680-9004-28ee5e52ece8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:29.850160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14ff78-79d3-4680-9004-28ee5e52ece8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:29.850678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14ff78-79d3-4680-9004-28ee5e52ece8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:29.892311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0f14ff78-79d3-4680-9004-28ee5e52ece8 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:29.893496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0f14ff78-79d3-4680-9004-28ee5e52ece8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:29.910561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:30.025204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f8692-901e-4415-8be3-ced78997499a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:30.031540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:30.032405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:30.032793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:30.033287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] PATH_INFO: `/v3/users/8bfc3658c16245f6ac3d06d25f9cfc31` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:30.033705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 975/1934] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:30:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:30.034612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bfc3658c16245f6ac3d06d25f9cfc31)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:30.036676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:30.092340 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bfc3658c16245f6ac3d06d25f9cfc31'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.094205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bfc3658c16245f6ac3d06d25f9cfc31'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.109279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bfc3658c16245f6ac3d06d25f9cfc31'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.117811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bfc3658c16245f6ac3d06d25f9cfc31'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.126445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bfc3658c16245f6ac3d06d25f9cfc31'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.133735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 960/1935] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:29 2022] DELETE /identity/v3/users/8bfc3658c16245f6ac3d06d25f9cfc31 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:30.150053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14ff78-79d3-4680-9004-28ee5e52ece8 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:30.152666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0f14ff78-79d3-4680-9004-28ee5e52ece8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:30.172237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14ff78-79d3-4680-9004-28ee5e52ece8 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:30.283170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2448c3d-b4ad-4fc0-967a-c4e824f4ecd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:30.283813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2448c3d-b4ad-4fc0-967a-c4e824f4ecd8 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:30.284094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2448c3d-b4ad-4fc0-967a-c4e824f4ecd8 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:30.284372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2448c3d-b4ad-4fc0-967a-c4e824f4ecd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:30.286653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f2448c3d-b4ad-4fc0-967a-c4e824f4ecd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:30.304791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2448c3d-b4ad-4fc0-967a-c4e824f4ecd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:30.306978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2448c3d-b4ad-4fc0-967a-c4e824f4ecd8 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:30.321021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 976/1936] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:30.348289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:30.350017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:30.437344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f20051-b3c9-4c83-8361-4303acadc77b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:30.442407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:30.443276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:30.443563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:30.443964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] PATH_INFO: `/v3/users/22fb5969f768467f9f691c15269abf62` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:30.445705 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22fb5969f768467f9f691c15269abf62)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:30.447721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:30.474463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:30.519044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22fb5969f768467f9f691c15269abf62'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.519894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22fb5969f768467f9f691c15269abf62'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.539744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22fb5969f768467f9f691c15269abf62'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.549020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22fb5969f768467f9f691c15269abf62'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.559144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22fb5969f768467f9f691c15269abf62'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:30.570390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 961/1937] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:30 2022] DELETE /identity/v3/users/22fb5969f768467f9f691c15269abf62 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:30.572669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2f68e-a9bd-4d78-9360-c838e7f5f0a3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:30.573057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2f68e-a9bd-4d78-9360-c838e7f5f0a3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:30.573458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2f68e-a9bd-4d78-9360-c838e7f5f0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:30.599096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-c2b2f68e-a9bd-4d78-9360-c838e7f5f0a3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:30.600639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c2b2f68e-a9bd-4d78-9360-c838e7f5f0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:30.674833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 977/1938] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:30.726003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:30.727517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:30.843647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783836f-6d94-4b8b-b34f-257df0e09957 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:30.857212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0583b4-2db8-4f81-b06c-228c4f6b9b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:30.858191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0583b4-2db8-4f81-b06c-228c4f6b9b27 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:30.858779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0583b4-2db8-4f81-b06c-228c4f6b9b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:30.859200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0583b4-2db8-4f81-b06c-228c4f6b9b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:30.882881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0583b4-2db8-4f81-b06c-228c4f6b9b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:30.885055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0583b4-2db8-4f81-b06c-228c4f6b9b27 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:30.897205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 962/1939] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:30.989356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2f68e-a9bd-4d78-9360-c838e7f5f0a3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:30.992768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c2b2f68e-a9bd-4d78-9360-c838e7f5f0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.032057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2f68e-a9bd-4d78-9360-c838e7f5f0a3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:31.044979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:31.046622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.047051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.048857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] PATH_INFO: `/v3/projects/116f1ab7b5af42c7a59a846d848de022` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.050836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=116f1ab7b5af42c7a59a846d848de022)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:31.054690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:31.115778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:31.162590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116f1ab7b5af42c7a59a846d848de022', 'request_id': 'req-ff44dd88-f083-4625-b782-3fec84d8f8b4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.171996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116f1ab7b5af42c7a59a846d848de022', 'request_id': 'req-ff44dd88-f083-4625-b782-3fec84d8f8b4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.180961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116f1ab7b5af42c7a59a846d848de022', 'request_id': 'req-ff44dd88-f083-4625-b782-3fec84d8f8b4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.197158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] The token cache is being invalidate because project 116f1ab7b5af42c7a59a846d848de022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:31.197688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.198472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.204683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 978/1940] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:30 2022] DELETE /identity/v3/projects/116f1ab7b5af42c7a59a846d848de022 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:31.207554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb83a9-05b1-4c60-8967-738a8a415c71 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.208288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb83a9-05b1-4c60-8967-738a8a415c71 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.208727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb83a9-05b1-4c60-8967-738a8a415c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.243627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-84eb83a9-05b1-4c60-8967-738a8a415c71 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:31.245216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-84eb83a9-05b1-4c60-8967-738a8a415c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.274757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:31.277098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.400043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 963/1941] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:31.407491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb83a9-05b1-4c60-8967-738a8a415c71 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:31.412947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-84eb83a9-05b1-4c60-8967-738a8a415c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.434161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44dd88-f083-4625-b782-3fec84d8f8b4 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:31.439276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb83a9-05b1-4c60-8967-738a8a415c71 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:31.443493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:31.444320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.445276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.445556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] PATH_INFO: `/v3/projects/7f57144b0ee84a34bd0b68ffcb97730e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.447640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f57144b0ee84a34bd0b68ffcb97730e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:31.447965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:31.448737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.449084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.449427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] PATH_INFO: `/v3/projects/bff421afdfb6422c8a60c80cc6591116` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.450598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:31.450862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff421afdfb6422c8a60c80cc6591116)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:31.454343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:31.512472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:31.537955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff421afdfb6422c8a60c80cc6591116', 'request_id': 'req-c667a6c7-e8ce-492c-8a31-a6f255010773'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.551576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff421afdfb6422c8a60c80cc6591116', 'request_id': 'req-c667a6c7-e8ce-492c-8a31-a6f255010773'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.564482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f57144b0ee84a34bd0b68ffcb97730e', 'request_id': 'req-31c67b80-d5cf-4524-bcf1-30514a1a866f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.578863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f57144b0ee84a34bd0b68ffcb97730e', 'request_id': 'req-31c67b80-d5cf-4524-bcf1-30514a1a866f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.587822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff421afdfb6422c8a60c80cc6591116', 'request_id': 'req-c667a6c7-e8ce-492c-8a31-a6f255010773'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.598094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f57144b0ee84a34bd0b68ffcb97730e', 'request_id': 'req-31c67b80-d5cf-4524-bcf1-30514a1a866f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.617352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] The token cache is being invalidate because project bff421afdfb6422c8a60c80cc6591116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:31.617819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.618633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-c667a6c7-e8ce-492c-8a31-a6f255010773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.628947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 964/1942] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:31 2022] DELETE /identity/v3/projects/bff421afdfb6422c8a60c80cc6591116 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:31.628947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c215e0-39a6-4ada-aa77-db6cf5aaba7e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.628947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c215e0-39a6-4ada-aa77-db6cf5aaba7e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.628947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c215e0-39a6-4ada-aa77-db6cf5aaba7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.632468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] The token cache is being invalidate because project 7f57144b0ee84a34bd0b68ffcb97730e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:31.640487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.648237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-31c67b80-d5cf-4524-bcf1-30514a1a866f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.653937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 979/1943] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:31 2022] DELETE /identity/v3/projects/7f57144b0ee84a34bd0b68ffcb97730e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:31.653937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f686a1-2b45-40b9-9bb9-d91bd4d9c852 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.653937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f686a1-2b45-40b9-9bb9-d91bd4d9c852 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.654216 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f686a1-2b45-40b9-9bb9-d91bd4d9c852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.685740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b4f686a1-2b45-40b9-9bb9-d91bd4d9c852 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:31.687476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b4f686a1-2b45-40b9-9bb9-d91bd4d9c852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.690666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-26c215e0-39a6-4ada-aa77-db6cf5aaba7e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:31.692404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-26c215e0-39a6-4ada-aa77-db6cf5aaba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.739553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:31.765387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 965/1944] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:31.766384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c215e0-39a6-4ada-aa77-db6cf5aaba7e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:31.767714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-26c215e0-39a6-4ada-aa77-db6cf5aaba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.771179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 980/1945] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:31.772604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dd12f-55c3-48a3-8632-d690ff3bd237 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.773000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dd12f-55c3-48a3-8632-d690ff3bd237 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.773373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dd12f-55c3-48a3-8632-d690ff3bd237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.783073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c215e0-39a6-4ada-aa77-db6cf5aaba7e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:31.789833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:31.790507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.790785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.791056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] PATH_INFO: `/v3/projects/d2616620915a49f3876922957c7c1364` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.792190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2616620915a49f3876922957c7c1364)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:31.796719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:31.805014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ec6dd12f-55c3-48a3-8632-d690ff3bd237 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:31.806235 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ec6dd12f-55c3-48a3-8632-d690ff3bd237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.816419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 981/1946] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:31.817266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dd12f-55c3-48a3-8632-d690ff3bd237 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:31.827329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dd12f-55c3-48a3-8632-d690ff3bd237 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:31.833047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdefbf-2c9d-4adf-b857-941b8f2a7067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:31.833821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdefbf-2c9d-4adf-b857-941b8f2a7067 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.834142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdefbf-2c9d-4adf-b857-941b8f2a7067 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.834404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdefbf-2c9d-4adf-b857-941b8f2a7067 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.834980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdefbf-2c9d-4adf-b857-941b8f2a7067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:31.835309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdefbf-2c9d-4adf-b857-941b8f2a7067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:31.837082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdefbf-2c9d-4adf-b857-941b8f2a7067 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:31.850032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 982/1947] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:31.851471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538eaffd-721f-40e3-b77f-3b62e9ffd15f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.851760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538eaffd-721f-40e3-b77f-3b62e9ffd15f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.852031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538eaffd-721f-40e3-b77f-3b62e9ffd15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.859027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2616620915a49f3876922957c7c1364', 'request_id': 'req-e2bc9922-ad19-415d-9d1a-5d6af6133d69'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.866067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2616620915a49f3876922957c7c1364', 'request_id': 'req-e2bc9922-ad19-415d-9d1a-5d6af6133d69'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.872331 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2616620915a49f3876922957c7c1364', 'request_id': 'req-e2bc9922-ad19-415d-9d1a-5d6af6133d69'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.883919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-538eaffd-721f-40e3-b77f-3b62e9ffd15f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:31.885259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-538eaffd-721f-40e3-b77f-3b62e9ffd15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.903966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] The token cache is being invalidate because project d2616620915a49f3876922957c7c1364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:31.905256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.908864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e2bc9922-ad19-415d-9d1a-5d6af6133d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:31.914771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 966/1948] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:31 2022] DELETE /identity/v3/projects/d2616620915a49f3876922957c7c1364 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:31.914771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b61547-4ac5-4f86-84b0-87b6df10d141 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.914771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b61547-4ac5-4f86-84b0-87b6df10d141 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.914771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b61547-4ac5-4f86-84b0-87b6df10d141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.941263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-90b61547-4ac5-4f86-84b0-87b6df10d141 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:31.943094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-90b61547-4ac5-4f86-84b0-87b6df10d141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.969053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 983/1949] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:31.969924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538eaffd-721f-40e3-b77f-3b62e9ffd15f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:31.971180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-538eaffd-721f-40e3-b77f-3b62e9ffd15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:31.986342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538eaffd-721f-40e3-b77f-3b62e9ffd15f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:31.992525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37e65d-442a-4407-bef3-61edc2396f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:31.993125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37e65d-442a-4407-bef3-61edc2396f6f admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:31.993403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37e65d-442a-4407-bef3-61edc2396f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:31.993674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d37e65d-442a-4407-bef3-61edc2396f6f admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:31.994310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37e65d-442a-4407-bef3-61edc2396f6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:31.997108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d37e65d-442a-4407-bef3-61edc2396f6f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.013323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:32.040216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 967/1950] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:32.041056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b61547-4ac5-4f86-84b0-87b6df10d141 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.042415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-90b61547-4ac5-4f86-84b0-87b6df10d141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:32.067711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 984/1951] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:31 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:32.067711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37e65d-442a-4407-bef3-61edc2396f6f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.070425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b61547-4ac5-4f86-84b0-87b6df10d141 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.147888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.149646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.149920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.150504 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] PATH_INFO: `/v3/users/00425f7cc6a643aba2c5dbc21a42b446` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.153226 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d37e65d-442a-4407-bef3-61edc2396f6f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.154543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=00425f7cc6a643aba2c5dbc21a42b446)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.157125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.182496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b11ab5-f9c4-4652-bb0a-40d9c7b4b5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.183384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b11ab5-f9c4-4652-bb0a-40d9c7b4b5df admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.183649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b11ab5-f9c4-4652-bb0a-40d9c7b4b5df admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.184075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b11ab5-f9c4-4652-bb0a-40d9c7b4b5df admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.184904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b11ab5-f9c4-4652-bb0a-40d9c7b4b5df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:32.185220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b11ab5-f9c4-4652-bb0a-40d9c7b4b5df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.188203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b11ab5-f9c4-4652-bb0a-40d9c7b4b5df admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.208122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 985/1952] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:32.208996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b11ab5-f9c4-4652-bb0a-40d9c7b4b5df admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.237297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b11ab5-f9c4-4652-bb0a-40d9c7b4b5df admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.246010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00425f7cc6a643aba2c5dbc21a42b446'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.247276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00425f7cc6a643aba2c5dbc21a42b446'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.249730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665229e-e83e-4077-b97f-ce863c0f9b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.250351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665229e-e83e-4077-b97f-ce863c0f9b1d admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.250627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665229e-e83e-4077-b97f-ce863c0f9b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.250897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665229e-e83e-4077-b97f-ce863c0f9b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.251918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665229e-e83e-4077-b97f-ce863c0f9b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:32.251918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665229e-e83e-4077-b97f-ce863c0f9b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.264303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665229e-e83e-4077-b97f-ce863c0f9b1d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.273329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00425f7cc6a643aba2c5dbc21a42b446'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.275080 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 986/1953] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:32.275080 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665229e-e83e-4077-b97f-ce863c0f9b1d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.287417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00425f7cc6a643aba2c5dbc21a42b446'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.289349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665229e-e83e-4077-b97f-ce863c0f9b1d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.296152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700674fa-957a-456f-a238-3568783ffa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.296931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700674fa-957a-456f-a238-3568783ffa3a admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.297184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700674fa-957a-456f-a238-3568783ffa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.297457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700674fa-957a-456f-a238-3568783ffa3a admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.298592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00425f7cc6a643aba2c5dbc21a42b446'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.298729 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700674fa-957a-456f-a238-3568783ffa3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.300734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700674fa-957a-456f-a238-3568783ffa3a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.306823 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 968/1954] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:32 2022] DELETE /identity/v3/users/00425f7cc6a643aba2c5dbc21a42b446 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:32.308098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.310314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:32.335640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 987/1955] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:32 2022] POST /identity/v3/users => generated 302 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:32.336999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700674fa-957a-456f-a238-3568783ffa3a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.342439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-700674fa-957a-456f-a238-3568783ffa3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:32.356995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700674fa-957a-456f-a238-3568783ffa3a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.362020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:32.409096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536d769-caae-4742-a5d7-3ac56143a80a admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.415269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e39dc-915c-41db-bde6-6ca16b3a4346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.416034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e39dc-915c-41db-bde6-6ca16b3a4346 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.416425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e39dc-915c-41db-bde6-6ca16b3a4346 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.416877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e39dc-915c-41db-bde6-6ca16b3a4346 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.417620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e39dc-915c-41db-bde6-6ca16b3a4346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.419822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e39dc-915c-41db-bde6-6ca16b3a4346 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.435991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:32.442815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c3d8f-9f47-4a33-9451-a50779924910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.443450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c3d8f-9f47-4a33-9451-a50779924910 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.443735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c3d8f-9f47-4a33-9451-a50779924910 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.444091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c3d8f-9f47-4a33-9451-a50779924910 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.444918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c3d8f-9f47-4a33-9451-a50779924910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.447251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c3d8f-9f47-4a33-9451-a50779924910 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.459264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 969/1956] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:32 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:32.460889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e39dc-915c-41db-bde6-6ca16b3a4346 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.464080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ce7e39dc-915c-41db-bde6-6ca16b3a4346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:32.480548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 988/1957] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:32 2022] POST /identity/v3/projects => generated 401 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:32.480548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c3d8f-9f47-4a33-9451-a50779924910 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.548635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e39dc-915c-41db-bde6-6ca16b3a4346 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.560090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c3d8f-9f47-4a33-9451-a50779924910 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.565386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3702-c89a-4a86-b683-bc32721a91fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.565987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3702-c89a-4a86-b683-bc32721a91fa admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.566263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3702-c89a-4a86-b683-bc32721a91fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.566600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c3702-c89a-4a86-b683-bc32721a91fa admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.567390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c3702-c89a-4a86-b683-bc32721a91fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.569411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c3702-c89a-4a86-b683-bc32721a91fa admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.596696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 989/1958] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:32 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:32.598012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3702-c89a-4a86-b683-bc32721a91fa admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.609475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c3702-c89a-4a86-b683-bc32721a91fa admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.613986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e8dbf-f4f9-4eb1-88fc-91ddddac47bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.614748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e8dbf-f4f9-4eb1-88fc-91ddddac47bd admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.615033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e8dbf-f4f9-4eb1-88fc-91ddddac47bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.615158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ff7bd-3e4e-4740-afd7-b2593ae01255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.615280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e8dbf-f4f9-4eb1-88fc-91ddddac47bd admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.615654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ff7bd-3e4e-4740-afd7-b2593ae01255 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.615932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ff7bd-3e4e-4740-afd7-b2593ae01255 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.616062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e8dbf-f4f9-4eb1-88fc-91ddddac47bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.616262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ff7bd-3e4e-4740-afd7-b2593ae01255 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.617048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ff7bd-3e4e-4740-afd7-b2593ae01255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.617619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e8dbf-f4f9-4eb1-88fc-91ddddac47bd admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.619094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ff7bd-3e4e-4740-afd7-b2593ae01255 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.632723 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 970/1959] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:32.634479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e8dbf-f4f9-4eb1-88fc-91ddddac47bd admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.649750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 990/1960] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:32 2022] POST /identity/v3/users => generated 301 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:32.649750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ff7bd-3e4e-4740-afd7-b2593ae01255 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.655444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e8dbf-f4f9-4eb1-88fc-91ddddac47bd admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.662290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ff7bd-3e4e-4740-afd7-b2593ae01255 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.663504 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0c83-3556-4dfb-8c2d-a0ef89df38de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.664404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0c83-3556-4dfb-8c2d-a0ef89df38de admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.664989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0c83-3556-4dfb-8c2d-a0ef89df38de admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.665471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0c83-3556-4dfb-8c2d-a0ef89df38de admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.666821 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b0c83-3556-4dfb-8c2d-a0ef89df38de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.668798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b0c83-3556-4dfb-8c2d-a0ef89df38de admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.691558 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66431d5-0994-4173-bd7b-4014e0052d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.692194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66431d5-0994-4173-bd7b-4014e0052d97 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.692477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66431d5-0994-4173-bd7b-4014e0052d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.692782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66431d5-0994-4173-bd7b-4014e0052d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.693441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66431d5-0994-4173-bd7b-4014e0052d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.695069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66431d5-0994-4173-bd7b-4014e0052d97 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.704167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 971/1961] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:32.705306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0c83-3556-4dfb-8c2d-a0ef89df38de admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.713200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 991/1962] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:32.713988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66431d5-0994-4173-bd7b-4014e0052d97 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.722937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0c83-3556-4dfb-8c2d-a0ef89df38de admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.735999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade22955-6c19-45eb-859c-efd76abdad5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.737215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade22955-6c19-45eb-859c-efd76abdad5e admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.740843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade22955-6c19-45eb-859c-efd76abdad5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.741128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade22955-6c19-45eb-859c-efd76abdad5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.741814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade22955-6c19-45eb-859c-efd76abdad5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.747571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade22955-6c19-45eb-859c-efd76abdad5e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.750662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66431d5-0994-4173-bd7b-4014e0052d97 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.772623 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 972/1963] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:32.773174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48b6f5-26c9-4fc9-ab4d-ef89a94b454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.773463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48b6f5-26c9-4fc9-ab4d-ef89a94b454b admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.773699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48b6f5-26c9-4fc9-ab4d-ef89a94b454b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.774054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48b6f5-26c9-4fc9-ab4d-ef89a94b454b admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.774720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48b6f5-26c9-4fc9-ab4d-ef89a94b454b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.775214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade22955-6c19-45eb-859c-efd76abdad5e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.776861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48b6f5-26c9-4fc9-ab4d-ef89a94b454b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.777502 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ade22955-6c19-45eb-859c-efd76abdad5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:32.800564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 992/1964] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:32.801493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48b6f5-26c9-4fc9-ab4d-ef89a94b454b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.803717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade22955-6c19-45eb-859c-efd76abdad5e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.809505 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.810901 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.812088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.815026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] PATH_INFO: `/v3/projects/f93cb372af3541819d6da5c3614ff81c` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.816442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93cb372af3541819d6da5c3614ff81c)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.819534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.822098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48b6f5-26c9-4fc9-ab4d-ef89a94b454b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:32.833162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25568eae-94e6-41a7-be61-be2e225c528d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:32.834056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25568eae-94e6-41a7-be61-be2e225c528d admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:32.834200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25568eae-94e6-41a7-be61-be2e225c528d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:32.834476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25568eae-94e6-41a7-be61-be2e225c528d admin admin] PATH_INFO: `/v3/projects/df8dcf90898c47459586f6d706709b51/users/cb91d181228044e9a85d5d518b4df088/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:32.861428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25568eae-94e6-41a7-be61-be2e225c528d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8dcf90898c47459586f6d706709b51, user_id=cb91d181228044e9a85d5d518b4df088, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:32.864917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25568eae-94e6-41a7-be61-be2e225c528d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:32.892146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93cb372af3541819d6da5c3614ff81c', 'request_id': 'req-0bf2a2f8-c4bd-4c63-b947-415470fd881f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.892943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 993/1965] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:32 2022] PUT /identity/v3/projects/df8dcf90898c47459586f6d706709b51/users/cb91d181228044e9a85d5d518b4df088/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:32.894625 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25568eae-94e6-41a7-be61-be2e225c528d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.899526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93cb372af3541819d6da5c3614ff81c', 'request_id': 'req-0bf2a2f8-c4bd-4c63-b947-415470fd881f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.916019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93cb372af3541819d6da5c3614ff81c', 'request_id': 'req-0bf2a2f8-c4bd-4c63-b947-415470fd881f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.920991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:32.928141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:32.938483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] The token cache is being invalidate because project f93cb372af3541819d6da5c3614ff81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:32.939091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.942144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:32.947080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 973/1966] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:32 2022] DELETE /identity/v3/projects/f93cb372af3541819d6da5c3614ff81c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:32.947080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:32.947080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:32.994245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25568eae-94e6-41a7-be61-be2e225c528d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.000815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d172c2-5393-4b71-bb20-8cbcedc99454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.001620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d172c2-5393-4b71-bb20-8cbcedc99454 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.002350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d172c2-5393-4b71-bb20-8cbcedc99454 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.002788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d172c2-5393-4b71-bb20-8cbcedc99454 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.003572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d172c2-5393-4b71-bb20-8cbcedc99454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.005645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d172c2-5393-4b71-bb20-8cbcedc99454 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.023095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 994/1967] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:32 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:33.024768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297def2-07b7-4120-a1b8-91987d372c35 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.025248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297def2-07b7-4120-a1b8-91987d372c35 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.025885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297def2-07b7-4120-a1b8-91987d372c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.046147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2a2f8-c4bd-4c63-b947-415470fd881f admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.053836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7276602-962a-4f1c-93ad-84fc88bdbe8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.054855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7276602-962a-4f1c-93ad-84fc88bdbe8d admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.055299 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7276602-962a-4f1c-93ad-84fc88bdbe8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.055734 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7276602-962a-4f1c-93ad-84fc88bdbe8d admin admin] PATH_INFO: `/v3/projects/2c5902fc93984117ba6d3dd8fdc0f6e0/users/7303f17b19d345e398de5ca6dad9060f/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.075743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7276602-962a-4f1c-93ad-84fc88bdbe8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5902fc93984117ba6d3dd8fdc0f6e0, user_id=7303f17b19d345e398de5ca6dad9060f, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.076953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b297def2-07b7-4120-a1b8-91987d372c35 None None] MFA Rules not processed for user `cb91d181228044e9a85d5d518b4df088`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:33.078477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b297def2-07b7-4120-a1b8-91987d372c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:33.079463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7276602-962a-4f1c-93ad-84fc88bdbe8d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.096730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 974/1968] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:32 2022] PUT /identity/v3/projects/2c5902fc93984117ba6d3dd8fdc0f6e0/users/7303f17b19d345e398de5ca6dad9060f/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:33.101651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7276602-962a-4f1c-93ad-84fc88bdbe8d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.103472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f7276602-962a-4f1c-93ad-84fc88bdbe8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:33.169177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 995/1969] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:33 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:33.169925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297def2-07b7-4120-a1b8-91987d372c35 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.171525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b297def2-07b7-4120-a1b8-91987d372c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:33.177173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7276602-962a-4f1c-93ad-84fc88bdbe8d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.181818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.182453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.182731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.183005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] PATH_INFO: `/v3/users/8e4cb151054942dbae158d8f504188c9` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.185049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e4cb151054942dbae158d8f504188c9)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.186007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297def2-07b7-4120-a1b8-91987d372c35 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.188328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.191809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d16c8-b789-4fc9-bd1b-865070f6ef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.192576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d16c8-b789-4fc9-bd1b-865070f6ef47 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.192922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d16c8-b789-4fc9-bd1b-865070f6ef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.193310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d16c8-b789-4fc9-bd1b-865070f6ef47 admin admin] PATH_INFO: `/v3/projects/bc2b5d19a520457eab36ee996b66f860/users/872c8c46b6ae423a8324dcac6a61f2cb/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.209681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d16c8-b789-4fc9-bd1b-865070f6ef47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc2b5d19a520457eab36ee996b66f860, user_id=872c8c46b6ae423a8324dcac6a61f2cb, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.212532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d16c8-b789-4fc9-bd1b-865070f6ef47 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.232211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 996/1970] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:33 2022] PUT /identity/v3/projects/bc2b5d19a520457eab36ee996b66f860/users/872c8c46b6ae423a8324dcac6a61f2cb/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:33.243506 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd981a9-2f14-4d23-965f-9955009fe428 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.244451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd981a9-2f14-4d23-965f-9955009fe428 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.244875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd981a9-2f14-4d23-965f-9955009fe428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.250745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e4cb151054942dbae158d8f504188c9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:33.251416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e4cb151054942dbae158d8f504188c9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:33.268747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e4cb151054942dbae158d8f504188c9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:33.279122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e4cb151054942dbae158d8f504188c9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:33.284192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-8bd981a9-2f14-4d23-965f-9955009fe428 None None] MFA Rules not processed for user `7303f17b19d345e398de5ca6dad9060f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:33.285798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8bd981a9-2f14-4d23-965f-9955009fe428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:33.301104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e4cb151054942dbae158d8f504188c9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:33.306577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:33.321934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 975/1971] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:33 2022] DELETE /identity/v3/users/8e4cb151054942dbae158d8f504188c9 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:33.323228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.340582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:33.384354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 997/1972] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:33.386025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8994ac-2dde-4cc3-93d7-19c522bb1b4a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.386623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8994ac-2dde-4cc3-93d7-19c522bb1b4a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.387123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8994ac-2dde-4cc3-93d7-19c522bb1b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.414828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:33.434113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-9a8994ac-2dde-4cc3-93d7-19c522bb1b4a None None] MFA Rules not processed for user `872c8c46b6ae423a8324dcac6a61f2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:33.436918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9a8994ac-2dde-4cc3-93d7-19c522bb1b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:33.481286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e784512-56ae-4f0c-9f7a-ce25bfca681c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.496815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4239a-6fac-48f1-9d9c-13df7684c7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.497455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4239a-6fac-48f1-9d9c-13df7684c7ef admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.497739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4239a-6fac-48f1-9d9c-13df7684c7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.498015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4239a-6fac-48f1-9d9c-13df7684c7ef admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.498660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4239a-6fac-48f1-9d9c-13df7684c7ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.500916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4239a-6fac-48f1-9d9c-13df7684c7ef admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.539683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 998/1973] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:33 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:33.541057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8994ac-2dde-4cc3-93d7-19c522bb1b4a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.542534 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9a8994ac-2dde-4cc3-93d7-19c522bb1b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:33.542672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 976/1974] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:33 2022] POST /identity/v3/projects => generated 409 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:33.547967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4239a-6fac-48f1-9d9c-13df7684c7ef admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.550139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-aea4239a-6fac-48f1-9d9c-13df7684c7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:33.561563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8994ac-2dde-4cc3-93d7-19c522bb1b4a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.647717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4239a-6fac-48f1-9d9c-13df7684c7ef admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.651753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba455d5-8ddf-4e97-bcbe-1d86567483f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.652867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba455d5-8ddf-4e97-bcbe-1d86567483f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.653245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba455d5-8ddf-4e97-bcbe-1d86567483f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.653641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba455d5-8ddf-4e97-bcbe-1d86567483f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.654793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba455d5-8ddf-4e97-bcbe-1d86567483f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.656816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8472c5-f771-4dca-871e-a98ae7cf1725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.658029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8472c5-f771-4dca-871e-a98ae7cf1725 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.658189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba455d5-8ddf-4e97-bcbe-1d86567483f9 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.658582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8472c5-f771-4dca-871e-a98ae7cf1725 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.659552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8472c5-f771-4dca-871e-a98ae7cf1725 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.660423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8472c5-f771-4dca-871e-a98ae7cf1725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.662969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8472c5-f771-4dca-871e-a98ae7cf1725 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.697913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 999/1975] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:33 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:33.699241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba455d5-8ddf-4e97-bcbe-1d86567483f9 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.720054 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 977/1976] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:33 2022] POST /identity/v3/projects => generated 399 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:33.720054 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8472c5-f771-4dca-871e-a98ae7cf1725 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.804945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8472c5-f771-4dca-871e-a98ae7cf1725 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.806203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba455d5-8ddf-4e97-bcbe-1d86567483f9 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.810969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464deaa-54c2-4b1a-85dd-2a8f21b560d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.811929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464deaa-54c2-4b1a-85dd-2a8f21b560d1 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.812279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464deaa-54c2-4b1a-85dd-2a8f21b560d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.812583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464deaa-54c2-4b1a-85dd-2a8f21b560d1 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.813302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464deaa-54c2-4b1a-85dd-2a8f21b560d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.815394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464deaa-54c2-4b1a-85dd-2a8f21b560d1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.816846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5076d7-9d01-46b4-8067-d0fee3b84f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.817851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5076d7-9d01-46b4-8067-d0fee3b84f44 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.818340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5076d7-9d01-46b4-8067-d0fee3b84f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.818815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5076d7-9d01-46b4-8067-d0fee3b84f44 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.819926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5076d7-9d01-46b4-8067-d0fee3b84f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.822100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5076d7-9d01-46b4-8067-d0fee3b84f44 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.835416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:33.845426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 978/1977] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:33 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:33.848923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464deaa-54c2-4b1a-85dd-2a8f21b560d1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.859442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1000/1978] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:33 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:33.859442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5076d7-9d01-46b4-8067-d0fee3b84f44 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.859765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464deaa-54c2-4b1a-85dd-2a8f21b560d1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.859842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ac5076d7-9d01-46b4-8067-d0fee3b84f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:33.864134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73e775-14a3-474a-8507-6a0d17abac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.864796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73e775-14a3-474a-8507-6a0d17abac69 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.865073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73e775-14a3-474a-8507-6a0d17abac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.865342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73e775-14a3-474a-8507-6a0d17abac69 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.865970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73e775-14a3-474a-8507-6a0d17abac69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.867996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73e775-14a3-474a-8507-6a0d17abac69 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.873410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5076d7-9d01-46b4-8067-d0fee3b84f44 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.882046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.882997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.883978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.884633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] PATH_INFO: `/v3/projects/bb4dcaf5d5cd47f48ea5271b437ddf41` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.886714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb4dcaf5d5cd47f48ea5271b437ddf41)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.888992 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.895884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 979/1979] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:33 2022] POST /identity/v3/users => generated 306 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:33.897194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73e775-14a3-474a-8507-6a0d17abac69 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.901411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:33.914605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73e775-14a3-474a-8507-6a0d17abac69 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.919682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9caf1-4f76-4404-b8f5-b4d3a7dc8701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:33.921327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9caf1-4f76-4404-b8f5-b4d3a7dc8701 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:33.921624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9caf1-4f76-4404-b8f5-b4d3a7dc8701 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:33.921894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9caf1-4f76-4404-b8f5-b4d3a7dc8701 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:33.922749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9caf1-4f76-4404-b8f5-b4d3a7dc8701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:33.925591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9caf1-4f76-4404-b8f5-b4d3a7dc8701 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:33.947985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 980/1980] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:33.951341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9caf1-4f76-4404-b8f5-b4d3a7dc8701 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:33.965330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9caf1-4f76-4404-b8f5-b4d3a7dc8701 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:33.970334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb4dcaf5d5cd47f48ea5271b437ddf41', 'request_id': 'req-5ccdda84-bb9a-4b70-be28-e93ba140df51'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:33.980104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb4dcaf5d5cd47f48ea5271b437ddf41', 'request_id': 'req-5ccdda84-bb9a-4b70-be28-e93ba140df51'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:33.988320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb4dcaf5d5cd47f48ea5271b437ddf41', 'request_id': 'req-5ccdda84-bb9a-4b70-be28-e93ba140df51'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:34.002708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] The token cache is being invalidate because project bb4dcaf5d5cd47f48ea5271b437ddf41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:34.003667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:34.003822 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:34.005835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1001/1981] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:33 2022] DELETE /identity/v3/projects/bb4dcaf5d5cd47f48ea5271b437ddf41 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:34.007020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.008742 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:34.038053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1863ab-1431-4fd2-939a-9911d158d062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.038672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1863ab-1431-4fd2-939a-9911d158d062 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.038950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1863ab-1431-4fd2-939a-9911d158d062 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.039220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1863ab-1431-4fd2-939a-9911d158d062 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.039856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1863ab-1431-4fd2-939a-9911d158d062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.041546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1863ab-1431-4fd2-939a-9911d158d062 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.052168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 981/1982] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:34.053130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1863ab-1431-4fd2-939a-9911d158d062 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.054831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4d1863ab-1431-4fd2-939a-9911d158d062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:34.068463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1863ab-1431-4fd2-939a-9911d158d062 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.073971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee8b4f-54bd-462b-b16a-72245ad77604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.074616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee8b4f-54bd-462b-b16a-72245ad77604 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.074850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee8b4f-54bd-462b-b16a-72245ad77604 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.075108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee8b4f-54bd-462b-b16a-72245ad77604 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.075869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee8b4f-54bd-462b-b16a-72245ad77604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.077450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee8b4f-54bd-462b-b16a-72245ad77604 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.087152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdda84-bb9a-4b70-be28-e93ba140df51 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.088358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 982/1983] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:34.089649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ec17c-3563-48dc-830e-e0c8d8e483b7 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.089930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ec17c-3563-48dc-830e-e0c8d8e483b7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.090207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ec17c-3563-48dc-830e-e0c8d8e483b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.093414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aad6fe-278a-4ce1-b74e-9fc3af9e83b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.094309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aad6fe-278a-4ce1-b74e-9fc3af9e83b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.094739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aad6fe-278a-4ce1-b74e-9fc3af9e83b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.095159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aad6fe-278a-4ce1-b74e-9fc3af9e83b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.095951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aad6fe-278a-4ce1-b74e-9fc3af9e83b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.098037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aad6fe-278a-4ce1-b74e-9fc3af9e83b4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.114681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1002/1984] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:34.114871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-355ec17c-3563-48dc-830e-e0c8d8e483b7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:34.114871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-355ec17c-3563-48dc-830e-e0c8d8e483b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:34.115038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aad6fe-278a-4ce1-b74e-9fc3af9e83b4 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.116622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c2aad6fe-278a-4ce1-b74e-9fc3af9e83b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:34.123023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 983/1985] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:34.124521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ec17c-3563-48dc-830e-e0c8d8e483b7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.137762 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ec17c-3563-48dc-830e-e0c8d8e483b7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.137987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aad6fe-278a-4ce1-b74e-9fc3af9e83b4 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.143029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bc6c8-e37e-4fec-816b-6a3d9415d9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.143165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3dd636-9ff9-477e-a38c-56c2b0c1ca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.143649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bc6c8-e37e-4fec-816b-6a3d9415d9e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.143753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dd636-9ff9-477e-a38c-56c2b0c1ca9a admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.143927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dd636-9ff9-477e-a38c-56c2b0c1ca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.144030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bc6c8-e37e-4fec-816b-6a3d9415d9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.144193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bc6c8-e37e-4fec-816b-6a3d9415d9e3 admin admin] PATH_INFO: `/v3/projects/cae782b9d24842f9a940ffbabf017e97/users/f6396078f04a40eab512463bece02d50/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.144296 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3dd636-9ff9-477e-a38c-56c2b0c1ca9a admin admin] PATH_INFO: `/v3/projects/7661bcddd62d48a8933795df55f5f74f/users/eb828a1741e04baa90a67bf8d79aed3a/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.160363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bc6c8-e37e-4fec-816b-6a3d9415d9e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae782b9d24842f9a940ffbabf017e97, user_id=f6396078f04a40eab512463bece02d50, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.163168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bc6c8-e37e-4fec-816b-6a3d9415d9e3 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.166146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dd636-9ff9-477e-a38c-56c2b0c1ca9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7661bcddd62d48a8933795df55f5f74f, user_id=eb828a1741e04baa90a67bf8d79aed3a, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.171639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3dd636-9ff9-477e-a38c-56c2b0c1ca9a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.183477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 984/1986] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:34 2022] PUT /identity/v3/projects/cae782b9d24842f9a940ffbabf017e97/users/f6396078f04a40eab512463bece02d50/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:34.186565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bc6c8-e37e-4fec-816b-6a3d9415d9e3 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.192292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1003/1987] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:34 2022] PUT /identity/v3/projects/7661bcddd62d48a8933795df55f5f74f/users/eb828a1741e04baa90a67bf8d79aed3a/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:34.192292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820ee7-38cc-4340-9db0-1f31189f9358 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.192292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820ee7-38cc-4340-9db0-1f31189f9358 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.192292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820ee7-38cc-4340-9db0-1f31189f9358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.213132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ee820ee7-38cc-4340-9db0-1f31189f9358 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:34.214426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ee820ee7-38cc-4340-9db0-1f31189f9358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:34.285626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bc6c8-e37e-4fec-816b-6a3d9415d9e3 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.291595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea717d9d-3fc6-4d81-8824-4766e9f7f1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.291749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1004/1988] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:34.293087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea717d9d-3fc6-4d81-8824-4766e9f7f1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.293212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daf906-a1dd-4070-b103-e75ec4e56dee None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.293456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daf906-a1dd-4070-b103-e75ec4e56dee None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.293732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36daf906-a1dd-4070-b103-e75ec4e56dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.294299 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea717d9d-3fc6-4d81-8824-4766e9f7f1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.294739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea717d9d-3fc6-4d81-8824-4766e9f7f1bc admin admin] PATH_INFO: `/v3/projects/a05ca708ba4e48daa4383b3e0174b04d/users/e93d66e6f79e4ee09881af5b9fa2bddd/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.320957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea717d9d-3fc6-4d81-8824-4766e9f7f1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05ca708ba4e48daa4383b3e0174b04d, user_id=e93d66e6f79e4ee09881af5b9fa2bddd, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.324022 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea717d9d-3fc6-4d81-8824-4766e9f7f1bc admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.326245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:34.342148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-36daf906-a1dd-4070-b103-e75ec4e56dee None None] MFA Rules not processed for user `f6396078f04a40eab512463bece02d50`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:34.344020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-36daf906-a1dd-4070-b103-e75ec4e56dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:34.370373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 985/1989] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:34 2022] PUT /identity/v3/projects/a05ca708ba4e48daa4383b3e0174b04d/users/e93d66e6f79e4ee09881af5b9fa2bddd/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:34.372714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea717d9d-3fc6-4d81-8824-4766e9f7f1bc admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.382341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:34.423117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1005/1990] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:34.424296 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36daf906-a1dd-4070-b103-e75ec4e56dee None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.484037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea717d9d-3fc6-4d81-8824-4766e9f7f1bc admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.499192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126aaa2-ef60-4899-86fa-2f5998dbafcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.500007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126aaa2-ef60-4899-86fa-2f5998dbafcc admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.500292 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126aaa2-ef60-4899-86fa-2f5998dbafcc admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.500588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126aaa2-ef60-4899-86fa-2f5998dbafcc admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.501244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6126aaa2-ef60-4899-86fa-2f5998dbafcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.502497 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36daf906-a1dd-4070-b103-e75ec4e56dee None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.505964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6126aaa2-ef60-4899-86fa-2f5998dbafcc admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.512696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba775cb9-90d7-4f35-b1bb-eade3120bd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.513652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba775cb9-90d7-4f35-b1bb-eade3120bd20 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.514114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba775cb9-90d7-4f35-b1bb-eade3120bd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.514683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba775cb9-90d7-4f35-b1bb-eade3120bd20 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.517116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba775cb9-90d7-4f35-b1bb-eade3120bd20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:34.517681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba775cb9-90d7-4f35-b1bb-eade3120bd20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.521213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba775cb9-90d7-4f35-b1bb-eade3120bd20 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.521928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 986/1991] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:34.522788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126aaa2-ef60-4899-86fa-2f5998dbafcc admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.535837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126aaa2-ef60-4899-86fa-2f5998dbafcc admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.539429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1006/1992] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:34.542515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba775cb9-90d7-4f35-b1bb-eade3120bd20 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.543538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c7355-6fae-443c-aeae-89158b28a009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.544464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c7355-6fae-443c-aeae-89158b28a009 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.545182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c7355-6fae-443c-aeae-89158b28a009 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.545660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c7355-6fae-443c-aeae-89158b28a009 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.546530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c7355-6fae-443c-aeae-89158b28a009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.549109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c7355-6fae-443c-aeae-89158b28a009 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.556562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba775cb9-90d7-4f35-b1bb-eade3120bd20 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.563153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b62938-5aee-4905-b003-36463bada534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.566811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b62938-5aee-4905-b003-36463bada534 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.566936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 987/1993] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:34.567017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b62938-5aee-4905-b003-36463bada534 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.567017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b62938-5aee-4905-b003-36463bada534 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.567166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c7355-6fae-443c-aeae-89158b28a009 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.567241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b62938-5aee-4905-b003-36463bada534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.568140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b62938-5aee-4905-b003-36463bada534 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.575466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c7355-6fae-443c-aeae-89158b28a009 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.582021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689c32e-bc51-42bb-bf00-cafcdb9087de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.582931 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c32e-bc51-42bb-bf00-cafcdb9087de admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.583241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c32e-bc51-42bb-bf00-cafcdb9087de admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.583518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689c32e-bc51-42bb-bf00-cafcdb9087de admin admin] PATH_INFO: `/v3/projects/7661bcddd62d48a8933795df55f5f74f/users/eb828a1741e04baa90a67bf8d79aed3a/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.586031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689c32e-bc51-42bb-bf00-cafcdb9087de admin admin] RBAC: Authorizing `identity:create_grant(project_id=7661bcddd62d48a8933795df55f5f74f, user_id=eb828a1741e04baa90a67bf8d79aed3a, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.588830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689c32e-bc51-42bb-bf00-cafcdb9087de admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.601606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 988/1994] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:34 2022] PUT /identity/v3/projects/7661bcddd62d48a8933795df55f5f74f/users/eb828a1741e04baa90a67bf8d79aed3a/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:34.602405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689c32e-bc51-42bb-bf00-cafcdb9087de admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.608156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1007/1995] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:34 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:34.610473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b62938-5aee-4905-b003-36463bada534 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.679817 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b62938-5aee-4905-b003-36463bada534 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.684867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c6011-1814-46a9-9e0c-4d18396156f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.685540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c6011-1814-46a9-9e0c-4d18396156f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.686972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c6011-1814-46a9-9e0c-4d18396156f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.686972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c6011-1814-46a9-9e0c-4d18396156f9 admin admin] PATH_INFO: `/v3/projects/a05ca708ba4e48daa4383b3e0174b04d/users/e93d66e6f79e4ee09881af5b9fa2bddd/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.689453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c6011-1814-46a9-9e0c-4d18396156f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05ca708ba4e48daa4383b3e0174b04d, user_id=e93d66e6f79e4ee09881af5b9fa2bddd, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.692205 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c6011-1814-46a9-9e0c-4d18396156f9 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.693217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689c32e-bc51-42bb-bf00-cafcdb9087de admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.697983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39731524-59e5-4516-bbfe-a63c71bf52f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.698556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39731524-59e5-4516-bbfe-a63c71bf52f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.698831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39731524-59e5-4516-bbfe-a63c71bf52f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.699348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39731524-59e5-4516-bbfe-a63c71bf52f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.699756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39731524-59e5-4516-bbfe-a63c71bf52f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.701663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39731524-59e5-4516-bbfe-a63c71bf52f2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.705696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1008/1996] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:34 2022] PUT /identity/v3/projects/a05ca708ba4e48daa4383b3e0174b04d/users/e93d66e6f79e4ee09881af5b9fa2bddd/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:34.709329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c6011-1814-46a9-9e0c-4d18396156f9 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.720809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:34.742895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 989/1997] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:34 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:34.742895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cf5a1-b966-425f-8ede-47b1135ef489 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.742895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cf5a1-b966-425f-8ede-47b1135ef489 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.742895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cf5a1-b966-425f-8ede-47b1135ef489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.772608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-922cf5a1-b966-425f-8ede-47b1135ef489 None None] MFA Rules not processed for user `eb828a1741e04baa90a67bf8d79aed3a`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:34.773868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-922cf5a1-b966-425f-8ede-47b1135ef489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:34.804570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c6011-1814-46a9-9e0c-4d18396156f9 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.825795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:34.852789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 990/1998] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:34 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:34.854385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a92e1-1e5c-4bf4-b355-294dd5579cc6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.854725 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a92e1-1e5c-4bf4-b355-294dd5579cc6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.855121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a92e1-1e5c-4bf4-b355-294dd5579cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.870021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4265-8f0d-4ba4-bc69-eec5f8d1b1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.870885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4265-8f0d-4ba4-bc69-eec5f8d1b1bf admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.871172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4265-8f0d-4ba4-bc69-eec5f8d1b1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.871449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a4265-8f0d-4ba4-bc69-eec5f8d1b1bf admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.872074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a4265-8f0d-4ba4-bc69-eec5f8d1b1bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.874633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a4265-8f0d-4ba4-bc69-eec5f8d1b1bf admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.885702 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-403a92e1-1e5c-4bf4-b355-294dd5579cc6 None None] MFA Rules not processed for user `e93d66e6f79e4ee09881af5b9fa2bddd`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:34.886906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-403a92e1-1e5c-4bf4-b355-294dd5579cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:34.900329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1009/1999] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:34 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:34.900329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4265-8f0d-4ba4-bc69-eec5f8d1b1bf admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.912055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a4265-8f0d-4ba4-bc69-eec5f8d1b1bf admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.917789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fec79-fd94-4c55-ad90-66f4cd7053eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.918411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fec79-fd94-4c55-ad90-66f4cd7053eb admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.918688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fec79-fd94-4c55-ad90-66f4cd7053eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.918953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fec79-fd94-4c55-ad90-66f4cd7053eb admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.919578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fec79-fd94-4c55-ad90-66f4cd7053eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.922369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fec79-fd94-4c55-ad90-66f4cd7053eb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:34.947018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1010/2000] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:34 2022] POST /identity/v3/users => generated 283 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:34.947018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fec79-fd94-4c55-ad90-66f4cd7053eb admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.947018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5a7fec79-fd94-4c55-ad90-66f4cd7053eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:34.968299 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 991/2001] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:34 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:34.969293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a92e1-1e5c-4bf4-b355-294dd5579cc6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:34.982585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a92e1-1e5c-4bf4-b355-294dd5579cc6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:34.989724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfafda1-3959-4b2d-bdff-a552e02eec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:34.990390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfafda1-3959-4b2d-bdff-a552e02eec1b admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:34.990666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfafda1-3959-4b2d-bdff-a552e02eec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:34.990938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfafda1-3959-4b2d-bdff-a552e02eec1b admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:34.991592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfafda1-3959-4b2d-bdff-a552e02eec1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:34.993271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfafda1-3959-4b2d-bdff-a552e02eec1b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.004289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 992/2002] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:35.005343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfafda1-3959-4b2d-bdff-a552e02eec1b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.015733 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfafda1-3959-4b2d-bdff-a552e02eec1b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.018993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fec79-fd94-4c55-ad90-66f4cd7053eb admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.021143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c1946-31ab-4472-94a8-608eef0c996b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.021750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c1946-31ab-4472-94a8-608eef0c996b admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.022028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c1946-31ab-4472-94a8-608eef0c996b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.022298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c1946-31ab-4472-94a8-608eef0c996b admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.022933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c1946-31ab-4472-94a8-608eef0c996b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.024248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2aad0-2f89-4a17-8f1c-9855c52c8c21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.024577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c1946-31ab-4472-94a8-608eef0c996b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.025408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2aad0-2f89-4a17-8f1c-9855c52c8c21 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.025987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2aad0-2f89-4a17-8f1c-9855c52c8c21 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.026404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2aad0-2f89-4a17-8f1c-9855c52c8c21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.036119 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 993/2003] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:35.036972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c1946-31ab-4472-94a8-608eef0c996b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.039262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2aad0-2f89-4a17-8f1c-9855c52c8c21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.040146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-c26c1946-31ab-4472-94a8-608eef0c996b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:35.041084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2aad0-2f89-4a17-8f1c-9855c52c8c21 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.052742 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1011/2004] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:34 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:35.054033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2aad0-2f89-4a17-8f1c-9855c52c8c21 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.054665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c1946-31ab-4472-94a8-608eef0c996b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.058428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66187b0-fed6-46b2-ad6b-504f059ef0a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.059098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66187b0-fed6-46b2-ad6b-504f059ef0a2 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.059517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66187b0-fed6-46b2-ad6b-504f059ef0a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.059942 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66187b0-fed6-46b2-ad6b-504f059ef0a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.067129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2aad0-2f89-4a17-8f1c-9855c52c8c21 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.072848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f2114-215c-4ddf-b6ab-4d61821f4f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.073846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f2114-215c-4ddf-b6ab-4d61821f4f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.074439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f2114-215c-4ddf-b6ab-4d61821f4f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.074951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f2114-215c-4ddf-b6ab-4d61821f4f88 admin admin] PATH_INFO: `/v3/projects/3a9ebf303399472d8b1aa1b420c436b4/users/2303f1922c27443fa977f61c2ebe2c16/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.075084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66187b0-fed6-46b2-ad6b-504f059ef0a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.077490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66187b0-fed6-46b2-ad6b-504f059ef0a2 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.091503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 994/2005] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:35 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:35.093072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66187b0-fed6-46b2-ad6b-504f059ef0a2 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.095687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f2114-215c-4ddf-b6ab-4d61821f4f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9ebf303399472d8b1aa1b420c436b4, user_id=2303f1922c27443fa977f61c2ebe2c16, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.099470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1f2114-215c-4ddf-b6ab-4d61821f4f88 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.105036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66187b0-fed6-46b2-ad6b-504f059ef0a2 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.117707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d84ba4-ac57-4f83-88dc-61e314820d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.118693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d84ba4-ac57-4f83-88dc-61e314820d38 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.119175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d84ba4-ac57-4f83-88dc-61e314820d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.119691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d84ba4-ac57-4f83-88dc-61e314820d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.120270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d84ba4-ac57-4f83-88dc-61e314820d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.122522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d84ba4-ac57-4f83-88dc-61e314820d38 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.128931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1012/2006] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:35 2022] PUT /identity/v3/projects/3a9ebf303399472d8b1aa1b420c436b4/users/2303f1922c27443fa977f61c2ebe2c16/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:35.133007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f2114-215c-4ddf-b6ab-4d61821f4f88 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.147060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 995/2007] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:35.148809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:35.154665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d84ba4-ac57-4f83-88dc-61e314820d38 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.178957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d84ba4-ac57-4f83-88dc-61e314820d38 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.269912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f2114-215c-4ddf-b6ab-4d61821f4f88 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.281008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d5760-6896-4e1c-b8a9-2dae6cb32594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.281812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d5760-6896-4e1c-b8a9-2dae6cb32594 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.282094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d5760-6896-4e1c-b8a9-2dae6cb32594 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.282368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d5760-6896-4e1c-b8a9-2dae6cb32594 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.283022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d5760-6896-4e1c-b8a9-2dae6cb32594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.283424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d0a83-b950-4d4f-9d00-188d2365a3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.284149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d0a83-b950-4d4f-9d00-188d2365a3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.284431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d0a83-b950-4d4f-9d00-188d2365a3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.284665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d5760-6896-4e1c-b8a9-2dae6cb32594 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.284792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d0a83-b950-4d4f-9d00-188d2365a3b3 admin admin] PATH_INFO: `/v3/projects/75a0f0ab7fbc4f05b8abb6d46af608cc/users/3aa9bc733466454ab436acfc580a4bb7/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.308280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d0a83-b950-4d4f-9d00-188d2365a3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a0f0ab7fbc4f05b8abb6d46af608cc, user_id=3aa9bc733466454ab436acfc580a4bb7, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.312447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d0a83-b950-4d4f-9d00-188d2365a3b3 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.317385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1013/2008] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:30:35.318462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d5760-6896-4e1c-b8a9-2dae6cb32594 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.370220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 996/2009] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:35 2022] PUT /identity/v3/projects/75a0f0ab7fbc4f05b8abb6d46af608cc/users/3aa9bc733466454ab436acfc580a4bb7/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:35.370220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d0a83-b950-4d4f-9d00-188d2365a3b3 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.388082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:35.428307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d5760-6896-4e1c-b8a9-2dae6cb32594 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.469512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d0a83-b950-4d4f-9d00-188d2365a3b3 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.476104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c444c65-416f-4cd3-ad0c-ac6a9850a1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.477318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c444c65-416f-4cd3-ad0c-ac6a9850a1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.477785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c444c65-416f-4cd3-ad0c-ac6a9850a1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.478448 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c444c65-416f-4cd3-ad0c-ac6a9850a1a4 admin admin] PATH_INFO: `/v3/projects/3a9ebf303399472d8b1aa1b420c436b4/users/2303f1922c27443fa977f61c2ebe2c16/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.480721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c444c65-416f-4cd3-ad0c-ac6a9850a1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9ebf303399472d8b1aa1b420c436b4, user_id=2303f1922c27443fa977f61c2ebe2c16, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.483695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c444c65-416f-4cd3-ad0c-ac6a9850a1a4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.502047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 997/2010] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:35 2022] PUT /identity/v3/projects/3a9ebf303399472d8b1aa1b420c436b4/users/2303f1922c27443fa977f61c2ebe2c16/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:35.503428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b389b-090a-4004-a496-dd80ee5afc4e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.504478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b389b-090a-4004-a496-dd80ee5afc4e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.505150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b389b-090a-4004-a496-dd80ee5afc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.513830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efda6c5-e5e8-4042-bf60-d674fb5e300f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.514602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efda6c5-e5e8-4042-bf60-d674fb5e300f service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.514999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efda6c5-e5e8-4042-bf60-d674fb5e300f service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.515487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efda6c5-e5e8-4042-bf60-d674fb5e300f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.518810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5efda6c5-e5e8-4042-bf60-d674fb5e300f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:35.584598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-817b389b-090a-4004-a496-dd80ee5afc4e None None] MFA Rules not processed for user `3aa9bc733466454ab436acfc580a4bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:35.586722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-817b389b-090a-4004-a496-dd80ee5afc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:35.646191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efda6c5-e5e8-4042-bf60-d674fb5e300f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.649071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efda6c5-e5e8-4042-bf60-d674fb5e300f service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.668272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1014/2011] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:35 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:35.671210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efda6c5-e5e8-4042-bf60-d674fb5e300f service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.690690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 998/2012] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:35.692120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efda6c5-e5e8-4042-bf60-d674fb5e300f service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:35.692606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c0bac-dbf3-4345-bb65-993e80dd0734 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.693123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c0bac-dbf3-4345-bb65-993e80dd0734 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.693870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c0bac-dbf3-4345-bb65-993e80dd0734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.706383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:35.745643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e14c0bac-dbf3-4345-bb65-993e80dd0734 None None] MFA Rules not processed for user `2303f1922c27443fa977f61c2ebe2c16`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:35.748431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e14c0bac-dbf3-4345-bb65-993e80dd0734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:35.815887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5f474-0a95-4c30-a737-55a29697df89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:35.817222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5f474-0a95-4c30-a737-55a29697df89 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:35.817222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5f474-0a95-4c30-a737-55a29697df89 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:35.817488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5f474-0a95-4c30-a737-55a29697df89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:35.819628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fda5f474-0a95-4c30-a737-55a29697df89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:35.934498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 999/2013] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:35.934498 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c0bac-dbf3-4345-bb65-993e80dd0734 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:35.948737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5f474-0a95-4c30-a737-55a29697df89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:35.951403 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5f474-0a95-4c30-a737-55a29697df89 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:35.965510 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1015/2014] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:35 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:36.005909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:36.083989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5f474-0a95-4c30-a737-55a29697df89 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:36.096364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5f474-0a95-4c30-a737-55a29697df89 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:36.098861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c0bac-dbf3-4345-bb65-993e80dd0734 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:36.103330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737481e-be9b-406c-8ad7-057d8b954dbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:36.104173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737481e-be9b-406c-8ad7-057d8b954dbf service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:36.104709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737481e-be9b-406c-8ad7-057d8b954dbf service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:36.105163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9737481e-be9b-406c-8ad7-057d8b954dbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:36.107288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9737481e-be9b-406c-8ad7-057d8b954dbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:36.112531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab58878-fbd4-41b6-8644-5a1ef567dfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:36.113182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab58878-fbd4-41b6-8644-5a1ef567dfa6 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:36.113457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab58878-fbd4-41b6-8644-5a1ef567dfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:36.113848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab58878-fbd4-41b6-8644-5a1ef567dfa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:36.114497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab58878-fbd4-41b6-8644-5a1ef567dfa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:36.116443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab58878-fbd4-41b6-8644-5a1ef567dfa6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:36.184944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1000/2015] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:35 2022] POST /identity/v3/projects => generated 367 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:36.185877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab58878-fbd4-41b6-8644-5a1ef567dfa6 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:36.263873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737481e-be9b-406c-8ad7-057d8b954dbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:36.266414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9737481e-be9b-406c-8ad7-057d8b954dbf service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:36.279446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1016/2016] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:36.320570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab58878-fbd4-41b6-8644-5a1ef567dfa6 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:36.325440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c69a13-bb80-4c32-9bb2-69a51d4a8388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:36.326052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c69a13-bb80-4c32-9bb2-69a51d4a8388 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:36.326333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c69a13-bb80-4c32-9bb2-69a51d4a8388 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:36.326604 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c69a13-bb80-4c32-9bb2-69a51d4a8388 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:36.327840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c69a13-bb80-4c32-9bb2-69a51d4a8388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:36.329839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c69a13-bb80-4c32-9bb2-69a51d4a8388 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:36.358602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1001/2017] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:36 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:36.359748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737481e-be9b-406c-8ad7-057d8b954dbf service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:36.372085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9737481e-be9b-406c-8ad7-057d8b954dbf service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:36.378441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1995720-7cf4-4246-bbed-cf8505493190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:36.379056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1995720-7cf4-4246-bbed-cf8505493190 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:36.379582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1995720-7cf4-4246-bbed-cf8505493190 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:36.379716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1995720-7cf4-4246-bbed-cf8505493190 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:36.380461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1995720-7cf4-4246-bbed-cf8505493190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:36.382212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1995720-7cf4-4246-bbed-cf8505493190 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:36.394046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1017/2018] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:36.398341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c69a13-bb80-4c32-9bb2-69a51d4a8388 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:36.421770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c69a13-bb80-4c32-9bb2-69a51d4a8388 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:36.429956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e762d-eee1-4475-8f0e-d3850ee14bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:36.430739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e762d-eee1-4475-8f0e-d3850ee14bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:36.431113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e762d-eee1-4475-8f0e-d3850ee14bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:36.431509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e762d-eee1-4475-8f0e-d3850ee14bb9 admin admin] PATH_INFO: `/v3/projects/4c431d86e3284d17937fdc4212e0a6da/users/8ecdd5d6c0d747b587638d47bdf68784/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:36.450523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e762d-eee1-4475-8f0e-d3850ee14bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c431d86e3284d17937fdc4212e0a6da, user_id=8ecdd5d6c0d747b587638d47bdf68784, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:36.454533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e762d-eee1-4475-8f0e-d3850ee14bb9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:36.488985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1002/2019] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:36 2022] PUT /identity/v3/projects/4c431d86e3284d17937fdc4212e0a6da/users/8ecdd5d6c0d747b587638d47bdf68784/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:36.491532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1995720-7cf4-4246-bbed-cf8505493190 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:36.504617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1995720-7cf4-4246-bbed-cf8505493190 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:36.570863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:36.583444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f69b79-e514-4afd-9ff1-103026870063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:36.584083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f69b79-e514-4afd-9ff1-103026870063 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:36.584362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f69b79-e514-4afd-9ff1-103026870063 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:36.584672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f69b79-e514-4afd-9ff1-103026870063 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:36.585314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f69b79-e514-4afd-9ff1-103026870063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:36.587160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f69b79-e514-4afd-9ff1-103026870063 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:36.600653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1018/2020] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:36.607745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e762d-eee1-4475-8f0e-d3850ee14bb9 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:36.621486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e762d-eee1-4475-8f0e-d3850ee14bb9 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:36.627154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e38a-f16e-444b-91fc-36b15adb146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:36.629398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e38a-f16e-444b-91fc-36b15adb146e admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:36.629398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e38a-f16e-444b-91fc-36b15adb146e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:36.629398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55e38a-f16e-444b-91fc-36b15adb146e admin admin] PATH_INFO: `/v3/projects/4c431d86e3284d17937fdc4212e0a6da/users/8ecdd5d6c0d747b587638d47bdf68784/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:36.630700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e38a-f16e-444b-91fc-36b15adb146e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c431d86e3284d17937fdc4212e0a6da, user_id=8ecdd5d6c0d747b587638d47bdf68784, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:36.633374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55e38a-f16e-444b-91fc-36b15adb146e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:36.645193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1003/2021] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:36 2022] PUT /identity/v3/projects/4c431d86e3284d17937fdc4212e0a6da/users/8ecdd5d6c0d747b587638d47bdf68784/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:36.656120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8eb31-1b93-45dd-ab45-c0a83e700872 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:36.656367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8eb31-1b93-45dd-ab45-c0a83e700872 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:36.656677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8eb31-1b93-45dd-ab45-c0a83e700872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:36.700007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-89c8eb31-1b93-45dd-ab45-c0a83e700872 None None] MFA Rules not processed for user `8ecdd5d6c0d747b587638d47bdf68784`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:36.704736 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-89c8eb31-1b93-45dd-ab45-c0a83e700872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:36.800625 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1019/2022] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:36.811736 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e38a-f16e-444b-91fc-36b15adb146e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:36.875613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55e38a-f16e-444b-91fc-36b15adb146e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:36.879260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848a45b-bcb5-41df-a6d5-bf006a933469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:36.879868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848a45b-bcb5-41df-a6d5-bf006a933469 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:36.880147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848a45b-bcb5-41df-a6d5-bf006a933469 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:36.880444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848a45b-bcb5-41df-a6d5-bf006a933469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:36.889618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848a45b-bcb5-41df-a6d5-bf006a933469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:36.891212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848a45b-bcb5-41df-a6d5-bf006a933469 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:36.896528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:36.903310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1004/2023] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:36.988009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8eb31-1b93-45dd-ab45-c0a83e700872 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:36.997709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8eb31-1b93-45dd-ab45-c0a83e700872 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:37.001448 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c937ed1-eccf-42a0-85f5-a8274e39ba6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:37.002052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c937ed1-eccf-42a0-85f5-a8274e39ba6a service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:37.002334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c937ed1-eccf-42a0-85f5-a8274e39ba6a service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:37.002592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c937ed1-eccf-42a0-85f5-a8274e39ba6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:37.065365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c937ed1-eccf-42a0-85f5-a8274e39ba6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:37.066919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c937ed1-eccf-42a0-85f5-a8274e39ba6a service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:37.075842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1020/2024] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:38.247097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848a45b-bcb5-41df-a6d5-bf006a933469 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:38.265959 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848a45b-bcb5-41df-a6d5-bf006a933469 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:38.270160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba4223-efb0-4a62-8418-85d725e0971d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:38.271000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba4223-efb0-4a62-8418-85d725e0971d service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:38.272691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba4223-efb0-4a62-8418-85d725e0971d service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:38.273072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba4223-efb0-4a62-8418-85d725e0971d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:38.344025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba4223-efb0-4a62-8418-85d725e0971d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:38.345625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba4223-efb0-4a62-8418-85d725e0971d service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:38.356300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1005/2025] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:42.516760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c937ed1-eccf-42a0-85f5-a8274e39ba6a service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:42.525342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c937ed1-eccf-42a0-85f5-a8274e39ba6a service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:42.529395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec39e07-ecd1-413e-9154-3a6fbfaf9159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:42.531165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec39e07-ecd1-413e-9154-3a6fbfaf9159 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:42.531721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec39e07-ecd1-413e-9154-3a6fbfaf9159 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:42.531721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec39e07-ecd1-413e-9154-3a6fbfaf9159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:42.599194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec39e07-ecd1-413e-9154-3a6fbfaf9159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:42.601590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec39e07-ecd1-413e-9154-3a6fbfaf9159 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:42.611696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1021/2026] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:42.996418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba4223-efb0-4a62-8418-85d725e0971d service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:43.065679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba4223-efb0-4a62-8418-85d725e0971d service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:43.070456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:43.071183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:43.071471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:43.071742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] PATH_INFO: `/v3/users/872c8c46b6ae423a8324dcac6a61f2cb` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:43.072955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=872c8c46b6ae423a8324dcac6a61f2cb)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:43.074959 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:43.134637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872c8c46b6ae423a8324dcac6a61f2cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.135303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872c8c46b6ae423a8324dcac6a61f2cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.149551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872c8c46b6ae423a8324dcac6a61f2cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.157826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872c8c46b6ae423a8324dcac6a61f2cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.166194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872c8c46b6ae423a8324dcac6a61f2cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.173648 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1006/2027] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:42 2022] DELETE /identity/v3/users/872c8c46b6ae423a8324dcac6a61f2cb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:43.179256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37cba8-d950-4e8f-9e94-bdc17d668fe7 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:43.179734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37cba8-d950-4e8f-9e94-bdc17d668fe7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:43.180105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37cba8-d950-4e8f-9e94-bdc17d668fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:43.194578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:43.200768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6e37cba8-d950-4e8f-9e94-bdc17d668fe7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:43.203187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6e37cba8-d950-4e8f-9e94-bdc17d668fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:43.266073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1022/2028] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:43.277370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:43.279159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:43.302295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:43.350031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32f9b-0890-444a-a719-21fb6f8d0a9c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:43.353946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b27c5-0762-46ec-910e-3f6258484d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:43.354566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b27c5-0762-46ec-910e-3f6258484d32 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:43.354915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b27c5-0762-46ec-910e-3f6258484d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:43.355128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b27c5-0762-46ec-910e-3f6258484d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:43.364695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b27c5-0762-46ec-910e-3f6258484d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:43.366151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b27c5-0762-46ec-910e-3f6258484d32 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:43.377058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1007/2029] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:43.817881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37cba8-d950-4e8f-9e94-bdc17d668fe7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:43.821263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6e37cba8-d950-4e8f-9e94-bdc17d668fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:43.845768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37cba8-d950-4e8f-9e94-bdc17d668fe7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:43.851303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:43.851994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:43.852207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:43.852842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] PATH_INFO: `/v3/projects/bc2b5d19a520457eab36ee996b66f860` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:43.854768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc2b5d19a520457eab36ee996b66f860)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:43.858502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:43.927712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc2b5d19a520457eab36ee996b66f860', 'request_id': 'req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.936032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc2b5d19a520457eab36ee996b66f860', 'request_id': 'req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.943313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc2b5d19a520457eab36ee996b66f860', 'request_id': 'req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.960846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] The token cache is being invalidate because project bc2b5d19a520457eab36ee996b66f860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:43.961512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.962195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:43.970835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b27c5-0762-46ec-910e-3f6258484d32 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:43.980597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1023/2030] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:43 2022] DELETE /identity/v3/projects/bc2b5d19a520457eab36ee996b66f860 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:43.988991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b27c5-0762-46ec-910e-3f6258484d32 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:44.064966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:44.065650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:44.065957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:44.066455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] PATH_INFO: `/v3/users/e93d66e6f79e4ee09881af5b9fa2bddd` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:44.067583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93d66e6f79e4ee09881af5b9fa2bddd)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:44.069584 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:44.138626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93d66e6f79e4ee09881af5b9fa2bddd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.140710 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93d66e6f79e4ee09881af5b9fa2bddd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.165873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93d66e6f79e4ee09881af5b9fa2bddd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.176851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93d66e6f79e4ee09881af5b9fa2bddd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.190860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b756dc4d-cf58-43b8-8eaf-f005e127d63a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93d66e6f79e4ee09881af5b9fa2bddd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.207024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1008/2031] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:43 2022] DELETE /identity/v3/users/e93d66e6f79e4ee09881af5b9fa2bddd => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:44.684427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:44.687424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:44.705182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ce569-af8a-4064-bf2c-6ce13f4b0ab7 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:44.778123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:44.778770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:44.779047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:44.779317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] PATH_INFO: `/v3/projects/a05ca708ba4e48daa4383b3e0174b04d` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:44.781078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a05ca708ba4e48daa4383b3e0174b04d)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:44.783065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:44.830066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:44.852717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a05ca708ba4e48daa4383b3e0174b04d', 'request_id': 'req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.859452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a05ca708ba4e48daa4383b3e0174b04d', 'request_id': 'req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.866589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a05ca708ba4e48daa4383b3e0174b04d', 'request_id': 'req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.886283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] The token cache is being invalidate because project a05ca708ba4e48daa4383b3e0174b04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:44.886648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.887634 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9c4f460f-41a8-4795-84c6-d2c4e57ae8b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:44.893841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1024/2032] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:44 2022] DELETE /identity/v3/projects/a05ca708ba4e48daa4383b3e0174b04d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:44.895864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa75f71-3ae2-4b67-b42d-61a64d46cc23 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:44.896513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa75f71-3ae2-4b67-b42d-61a64d46cc23 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:44.896982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa75f71-3ae2-4b67-b42d-61a64d46cc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:44.925987 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-baa75f71-3ae2-4b67-b42d-61a64d46cc23 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:44.927259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-baa75f71-3ae2-4b67-b42d-61a64d46cc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:44.972191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:45.010079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1009/2033] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:45.018679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eb1ea-71e5-49e9-9ed7-f019d72cd254 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.018987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eb1ea-71e5-49e9-9ed7-f019d72cd254 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.019257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eb1ea-71e5-49e9-9ed7-f019d72cd254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.027243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa75f71-3ae2-4b67-b42d-61a64d46cc23 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.028825 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-baa75f71-3ae2-4b67-b42d-61a64d46cc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:45.041317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa75f71-3ae2-4b67-b42d-61a64d46cc23 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.043197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ff1eb1ea-71e5-49e9-9ed7-f019d72cd254 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:45.044613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ff1eb1ea-71e5-49e9-9ed7-f019d72cd254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:45.047728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.048866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.049289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.049733 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] PATH_INFO: `/v3/users/3aa9bc733466454ab436acfc580a4bb7` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.051018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa9bc733466454ab436acfc580a4bb7)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.053058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1025/2034] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:45.053247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.057554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eb1ea-71e5-49e9-9ed7-f019d72cd254 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.067974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eb1ea-71e5-49e9-9ed7-f019d72cd254 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.072656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2284184-1e8e-4d9b-97ca-8fb40555f2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.073263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2284184-1e8e-4d9b-97ca-8fb40555f2ed admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.073537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2284184-1e8e-4d9b-97ca-8fb40555f2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.073809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2284184-1e8e-4d9b-97ca-8fb40555f2ed admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.074456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2284184-1e8e-4d9b-97ca-8fb40555f2ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:45.074758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2284184-1e8e-4d9b-97ca-8fb40555f2ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.077182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2284184-1e8e-4d9b-97ca-8fb40555f2ed admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.091979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1026/2035] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:30:45.097472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2284184-1e8e-4d9b-97ca-8fb40555f2ed admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.110869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2284184-1e8e-4d9b-97ca-8fb40555f2ed admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.113519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa9bc733466454ab436acfc580a4bb7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.114208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa9bc733466454ab436acfc580a4bb7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.119107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d67e57-252c-46cd-94d8-27c5461869df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.120682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d67e57-252c-46cd-94d8-27c5461869df admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.121133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d67e57-252c-46cd-94d8-27c5461869df admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.121713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d67e57-252c-46cd-94d8-27c5461869df admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.122977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d67e57-252c-46cd-94d8-27c5461869df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.126352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d67e57-252c-46cd-94d8-27c5461869df admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.127276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa9bc733466454ab436acfc580a4bb7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.134344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa9bc733466454ab436acfc580a4bb7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.143581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-14a22cea-6432-4c0c-8025-d98870a5f4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa9bc733466454ab436acfc580a4bb7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.153664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1010/2036] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:45 2022] DELETE /identity/v3/users/3aa9bc733466454ab436acfc580a4bb7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:45.157908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc027a-bccf-47d0-9d70-f0851cd402e7 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.158355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc027a-bccf-47d0-9d70-f0851cd402e7 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.158633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc027a-bccf-47d0-9d70-f0851cd402e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.161350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1027/2037] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:45 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:30:45.165765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d67e57-252c-46cd-94d8-27c5461869df admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.168829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c4d67e57-252c-46cd-94d8-27c5461869df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:45.178664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-89dc027a-bccf-47d0-9d70-f0851cd402e7 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:45.179857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-89dc027a-bccf-47d0-9d70-f0851cd402e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:45.265699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1011/2038] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:45.266667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc027a-bccf-47d0-9d70-f0851cd402e7 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.268294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-89dc027a-bccf-47d0-9d70-f0851cd402e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:45.270788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d67e57-252c-46cd-94d8-27c5461869df admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.278146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ceb31-ae39-47fa-873e-a3f2db9146a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.279002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ceb31-ae39-47fa-873e-a3f2db9146a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.279414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ceb31-ae39-47fa-873e-a3f2db9146a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.280010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ceb31-ae39-47fa-873e-a3f2db9146a1 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.281597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ceb31-ae39-47fa-873e-a3f2db9146a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.285370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ceb31-ae39-47fa-873e-a3f2db9146a1 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.316593 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1028/2039] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:45 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:30:45.320521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ceb31-ae39-47fa-873e-a3f2db9146a1 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.335771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ceb31-ae39-47fa-873e-a3f2db9146a1 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.349011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1defc-b7b5-4a45-b76d-7f55e5c1cb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.350140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1defc-b7b5-4a45-b76d-7f55e5c1cb4c admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.350140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1defc-b7b5-4a45-b76d-7f55e5c1cb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.350332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1defc-b7b5-4a45-b76d-7f55e5c1cb4c admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.350994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1defc-b7b5-4a45-b76d-7f55e5c1cb4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.352680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1defc-b7b5-4a45-b76d-7f55e5c1cb4c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.370127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1029/2040] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:45.371080 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1defc-b7b5-4a45-b76d-7f55e5c1cb4c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.383191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1defc-b7b5-4a45-b76d-7f55e5c1cb4c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.389178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc027a-bccf-47d0-9d70-f0851cd402e7 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.393798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de5eb-829b-426c-af46-f6b831d8fa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.394354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de5eb-829b-426c-af46-f6b831d8fa15 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.395106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de5eb-829b-426c-af46-f6b831d8fa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.395106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de5eb-829b-426c-af46-f6b831d8fa15 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.395571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720de5eb-829b-426c-af46-f6b831d8fa15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.396600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860d6a0-6ce1-4252-a323-277fd356c504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.397217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720de5eb-829b-426c-af46-f6b831d8fa15 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.397565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860d6a0-6ce1-4252-a323-277fd356c504 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.397985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860d6a0-6ce1-4252-a323-277fd356c504 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.398397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860d6a0-6ce1-4252-a323-277fd356c504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.409945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:45.417686 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860d6a0-6ce1-4252-a323-277fd356c504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.419538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1030/2041] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:45.424958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de5eb-829b-426c-af46-f6b831d8fa15 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.428170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860d6a0-6ce1-4252-a323-277fd356c504 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.443903 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de5eb-829b-426c-af46-f6b831d8fa15 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.448326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1012/2042] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:45.452568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0d3be5-f5a9-47a4-9949-28334429a43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.453953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d3be5-f5a9-47a4-9949-28334429a43f admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.454372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d3be5-f5a9-47a4-9949-28334429a43f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.455382 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0d3be5-f5a9-47a4-9949-28334429a43f admin admin] PATH_INFO: `/v3/projects/76b9ccd09b574daba079f9e610970939/users/e1dc9c62ddd44b53a51d625c5b81331d/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.489463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d3be5-f5a9-47a4-9949-28334429a43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b9ccd09b574daba079f9e610970939, user_id=e1dc9c62ddd44b53a51d625c5b81331d, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.496225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0d3be5-f5a9-47a4-9949-28334429a43f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.512968 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860d6a0-6ce1-4252-a323-277fd356c504 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.517192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4860d6a0-6ce1-4252-a323-277fd356c504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:45.533758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1031/2043] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:45 2022] PUT /identity/v3/projects/76b9ccd09b574daba079f9e610970939/users/e1dc9c62ddd44b53a51d625c5b81331d/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:45.541200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b034f85-5d5e-4c81-8410-9064de427fde None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.541978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b034f85-5d5e-4c81-8410-9064de427fde None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.542752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b034f85-5d5e-4c81-8410-9064de427fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.564815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860d6a0-6ce1-4252-a323-277fd356c504 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.582620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0b034f85-5d5e-4c81-8410-9064de427fde None None] MFA Rules not processed for user `e1dc9c62ddd44b53a51d625c5b81331d`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:45.584141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0b034f85-5d5e-4c81-8410-9064de427fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:45.594636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:45.687639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.688791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.689652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.689652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] PATH_INFO: `/v3/users/cb91d181228044e9a85d5d518b4df088` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.700862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb91d181228044e9a85d5d518b4df088)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.708218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.776786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1032/2044] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:45.782597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b034f85-5d5e-4c81-8410-9064de427fde None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.796210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b034f85-5d5e-4c81-8410-9064de427fde None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:45.802294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27c864-41da-4b36-8fa6-790cc87e6629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:45.803032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27c864-41da-4b36-8fa6-790cc87e6629 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.803466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27c864-41da-4b36-8fa6-790cc87e6629 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.804077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27c864-41da-4b36-8fa6-790cc87e6629 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.807319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb91d181228044e9a85d5d518b4df088'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.808202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb91d181228044e9a85d5d518b4df088'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.817151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27c864-41da-4b36-8fa6-790cc87e6629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:45.821311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27c864-41da-4b36-8fa6-790cc87e6629 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:45.826458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb91d181228044e9a85d5d518b4df088'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.836152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb91d181228044e9a85d5d518b4df088'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.849675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-71530284-02f4-4184-a37a-61ef42f01395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb91d181228044e9a85d5d518b4df088'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:45.859399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1013/2045] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:45 2022] DELETE /identity/v3/users/cb91d181228044e9a85d5d518b4df088 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:45.872807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b8bea-2116-4f0b-9359-3375c5238f62 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:45.875058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b8bea-2116-4f0b-9359-3375c5238f62 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:45.877359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b8bea-2116-4f0b-9359-3375c5238f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:45.900039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1033/2046] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:45 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:45.901633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27c864-41da-4b36-8fa6-790cc87e6629 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:45.908338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3b27c864-41da-4b36-8fa6-790cc87e6629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:45.926451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-379b8bea-2116-4f0b-9359-3375c5238f62 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:45.927732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-379b8bea-2116-4f0b-9359-3375c5238f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:46.074731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1014/2047] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:46.095041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b8bea-2116-4f0b-9359-3375c5238f62 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:46.097477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-379b8bea-2116-4f0b-9359-3375c5238f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:46.120677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:46.145788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27c864-41da-4b36-8fa6-790cc87e6629 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:46.168297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff5425-1052-4883-995a-91baef5a1a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:46.181543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff5425-1052-4883-995a-91baef5a1a38 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.182333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff5425-1052-4883-995a-91baef5a1a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.183928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff5425-1052-4883-995a-91baef5a1a38 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.186600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff5425-1052-4883-995a-91baef5a1a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:46.193147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b8bea-2116-4f0b-9359-3375c5238f62 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:46.196166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff5425-1052-4883-995a-91baef5a1a38 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:46.197182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0336d-cea8-4075-87b6-f749bc533b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:46.197871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0336d-cea8-4075-87b6-f749bc533b5a service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.198150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0336d-cea8-4075-87b6-f749bc533b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.198423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0336d-cea8-4075-87b6-f749bc533b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.210995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0336d-cea8-4075-87b6-f749bc533b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:46.212548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0336d-cea8-4075-87b6-f749bc533b5a service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:46.225254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1015/2048] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:30:46.233351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0336d-cea8-4075-87b6-f749bc533b5a service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:46.238341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7cc0336d-cea8-4075-87b6-f749bc533b5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:46.241749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1034/2049] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:45 2022] POST /identity/v3/users => generated 294 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:46.245798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff5425-1052-4883-995a-91baef5a1a38 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:46.259541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff5425-1052-4883-995a-91baef5a1a38 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:46.264447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0336d-cea8-4075-87b6-f749bc533b5a service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:46.270319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45d127-30c5-45b6-9213-2a5c176aef7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:46.270956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45d127-30c5-45b6-9213-2a5c176aef7d admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.271235 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45d127-30c5-45b6-9213-2a5c176aef7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.271505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45d127-30c5-45b6-9213-2a5c176aef7d admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.272168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45d127-30c5-45b6-9213-2a5c176aef7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:46.273885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45d127-30c5-45b6-9213-2a5c176aef7d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:46.275808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:46.276818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.277338 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.277843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] PATH_INFO: `/v3/projects/75a0f0ab7fbc4f05b8abb6d46af608cc` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.279704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a0f0ab7fbc4f05b8abb6d46af608cc)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:46.282857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:46.287298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1035/2050] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:30:46.293919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45d127-30c5-45b6-9213-2a5c176aef7d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:46.307693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45d127-30c5-45b6-9213-2a5c176aef7d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:46.324905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f549eaf-01f1-474f-bb42-e032a6dd780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:46.328757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f549eaf-01f1-474f-bb42-e032a6dd780d admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.329323 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f549eaf-01f1-474f-bb42-e032a6dd780d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.329739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f549eaf-01f1-474f-bb42-e032a6dd780d admin admin] PATH_INFO: `/v3/projects/db93bdd05f964880a0d7f84977bc199d/users/c490084fe6e74cdbaa024b0bb465df9c/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.374938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f549eaf-01f1-474f-bb42-e032a6dd780d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db93bdd05f964880a0d7f84977bc199d, user_id=c490084fe6e74cdbaa024b0bb465df9c, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:46.378052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f549eaf-01f1-474f-bb42-e032a6dd780d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:46.391265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a0f0ab7fbc4f05b8abb6d46af608cc', 'request_id': 'req-9a718d3c-6cd5-47c5-b292-57fb2531b56a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.401985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1036/2051] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:46 2022] PUT /identity/v3/projects/db93bdd05f964880a0d7f84977bc199d/users/c490084fe6e74cdbaa024b0bb465df9c/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:46.402235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a0f0ab7fbc4f05b8abb6d46af608cc', 'request_id': 'req-9a718d3c-6cd5-47c5-b292-57fb2531b56a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.408041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f549eaf-01f1-474f-bb42-e032a6dd780d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:46.413571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a0f0ab7fbc4f05b8abb6d46af608cc', 'request_id': 'req-9a718d3c-6cd5-47c5-b292-57fb2531b56a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.431213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] The token cache is being invalidate because project 75a0f0ab7fbc4f05b8abb6d46af608cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:46.431512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.432187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.438539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1016/2052] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:46 2022] DELETE /identity/v3/projects/75a0f0ab7fbc4f05b8abb6d46af608cc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:46.439571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:46.441654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:46.497480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:46.529787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f549eaf-01f1-474f-bb42-e032a6dd780d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:46.536441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0bd7b-650c-49ed-a270-b36600a4edbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:46.543555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0bd7b-650c-49ed-a270-b36600a4edbe admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.543845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0bd7b-650c-49ed-a270-b36600a4edbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.544116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0bd7b-650c-49ed-a270-b36600a4edbe admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.549652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0bd7b-650c-49ed-a270-b36600a4edbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:46.552223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0bd7b-650c-49ed-a270-b36600a4edbe admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:46.555826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718d3c-6cd5-47c5-b292-57fb2531b56a admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:46.561784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:46.562681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.563185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.563570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] PATH_INFO: `/v3/users/8ecdd5d6c0d747b587638d47bdf68784` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.564986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ecdd5d6c0d747b587638d47bdf68784)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:46.567565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1037/2053] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:46.572146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0bd7b-650c-49ed-a270-b36600a4edbe admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:46.574529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d6c0bd7b-650c-49ed-a270-b36600a4edbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:46.575973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:46.597570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0bd7b-650c-49ed-a270-b36600a4edbe admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:46.602620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a562a-8a3a-43fe-92ef-abcbc569817b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:46.603413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a562a-8a3a-43fe-92ef-abcbc569817b admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.603764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a562a-8a3a-43fe-92ef-abcbc569817b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.604044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a562a-8a3a-43fe-92ef-abcbc569817b admin admin] PATH_INFO: `/v3/projects/db93bdd05f964880a0d7f84977bc199d/users/c490084fe6e74cdbaa024b0bb465df9c/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.608734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a562a-8a3a-43fe-92ef-abcbc569817b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db93bdd05f964880a0d7f84977bc199d, user_id=c490084fe6e74cdbaa024b0bb465df9c, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:46.609201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a562a-8a3a-43fe-92ef-abcbc569817b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:46.648931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1038/2054] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:46 2022] PUT /identity/v3/projects/db93bdd05f964880a0d7f84977bc199d/users/c490084fe6e74cdbaa024b0bb465df9c/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:46.667161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734640d3-a3bc-49e8-a98b-a5f7271dc331 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.667818 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734640d3-a3bc-49e8-a98b-a5f7271dc331 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.668175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734640d3-a3bc-49e8-a98b-a5f7271dc331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.674405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:46.681793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ecdd5d6c0d747b587638d47bdf68784'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.683231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ecdd5d6c0d747b587638d47bdf68784'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.708710 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ecdd5d6c0d747b587638d47bdf68784'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.719351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-734640d3-a3bc-49e8-a98b-a5f7271dc331 None None] MFA Rules not processed for user `c490084fe6e74cdbaa024b0bb465df9c`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:46.720135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ecdd5d6c0d747b587638d47bdf68784'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.721174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-734640d3-a3bc-49e8-a98b-a5f7271dc331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:46.732815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ecdd5d6c0d747b587638d47bdf68784'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:46.744429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1017/2055] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:46 2022] DELETE /identity/v3/users/8ecdd5d6c0d747b587638d47bdf68784 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:46.804556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:46.811234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:46.849171 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1039/2056] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:46.859255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734640d3-a3bc-49e8-a98b-a5f7271dc331 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:46.861968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-734640d3-a3bc-49e8-a98b-a5f7271dc331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:46.915051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eddee6-89f2-4664-8743-ff21d9afe556 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:46.928454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:46.929196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:46.929479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:46.931599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] PATH_INFO: `/v3/projects/df8dcf90898c47459586f6d706709b51` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:46.932929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8dcf90898c47459586f6d706709b51)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:46.935429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:46.962776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734640d3-a3bc-49e8-a98b-a5f7271dc331 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.014382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8dcf90898c47459586f6d706709b51', 'request_id': 'req-1f428d3e-074e-4766-827d-face40182d28'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.023908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8dcf90898c47459586f6d706709b51', 'request_id': 'req-1f428d3e-074e-4766-827d-face40182d28'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.031908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8dcf90898c47459586f6d706709b51', 'request_id': 'req-1f428d3e-074e-4766-827d-face40182d28'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.036200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.036818 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.037082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.037432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.040202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.046912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] The token cache is being invalidate because project df8dcf90898c47459586f6d706709b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:47.047335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.048111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.050489 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1018/2057] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:46 2022] DELETE /identity/v3/projects/df8dcf90898c47459586f6d706709b51 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:47.054867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:47.056378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.072850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:47.119028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:47.132199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.134094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:47.143451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f428d3e-074e-4766-827d-face40182d28 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.146839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1040/2058] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:47.151925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.151925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.151925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.151925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] PATH_INFO: `/v3/users/eb828a1741e04baa90a67bf8d79aed3a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.156073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:47.157571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb828a1741e04baa90a67bf8d79aed3a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.157729 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.160916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:47.175545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38427a-4dad-4980-8069-0df9d1ace550 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.187288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.187895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.188254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.188441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] PATH_INFO: `/v3/projects/4c431d86e3284d17937fdc4212e0a6da` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.189728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c431d86e3284d17937fdc4212e0a6da)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.191663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:47.229452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb828a1741e04baa90a67bf8d79aed3a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.230190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb828a1741e04baa90a67bf8d79aed3a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.266022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c431d86e3284d17937fdc4212e0a6da', 'request_id': 'req-3eef30dc-a475-4463-93d0-0518e78f9deb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.269048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb828a1741e04baa90a67bf8d79aed3a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.272737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c431d86e3284d17937fdc4212e0a6da', 'request_id': 'req-3eef30dc-a475-4463-93d0-0518e78f9deb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.282276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c431d86e3284d17937fdc4212e0a6da', 'request_id': 'req-3eef30dc-a475-4463-93d0-0518e78f9deb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.283536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb828a1741e04baa90a67bf8d79aed3a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.296220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb828a1741e04baa90a67bf8d79aed3a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.301173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] The token cache is being invalidate because project 4c431d86e3284d17937fdc4212e0a6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:47.301681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.302540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3eef30dc-a475-4463-93d0-0518e78f9deb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.308217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1019/2059] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:47 2022] DELETE /identity/v3/users/eb828a1741e04baa90a67bf8d79aed3a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:47.309852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1041/2060] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:47 2022] DELETE /identity/v3/projects/4c431d86e3284d17937fdc4212e0a6da => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:47.311156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:47.313652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.315720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ca746-1492-49b5-b245-b8f8d804e908 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.316012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ca746-1492-49b5-b245-b8f8d804e908 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.316296 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ca746-1492-49b5-b245-b8f8d804e908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.345220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-7d3ca746-1492-49b5-b245-b8f8d804e908 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:47.346840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7d3ca746-1492-49b5-b245-b8f8d804e908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.402950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b1d7f-7128-4bb2-b951-301786ec432d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.408929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.409616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.409898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.410206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] PATH_INFO: `/v3/users/7303f17b19d345e398de5ca6dad9060f` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.411441 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7303f17b19d345e398de5ca6dad9060f)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.413618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:47.471771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1042/2061] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:47.474386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf5fd6-06cb-4cd7-b72c-1cf50d328b30 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.475156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf5fd6-06cb-4cd7-b72c-1cf50d328b30 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.475588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf5fd6-06cb-4cd7-b72c-1cf50d328b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.476229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7303f17b19d345e398de5ca6dad9060f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.477054 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7303f17b19d345e398de5ca6dad9060f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.510014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-2bbf5fd6-06cb-4cd7-b72c-1cf50d328b30 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:47.511237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2bbf5fd6-06cb-4cd7-b72c-1cf50d328b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.517551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7303f17b19d345e398de5ca6dad9060f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.525030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:47.530149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7303f17b19d345e398de5ca6dad9060f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.536332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1043/2062] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:47.541760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf5fd6-06cb-4cd7-b72c-1cf50d328b30 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:47.547017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5e8df3fc-0e75-4d01-8b96-83fa82aeee81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7303f17b19d345e398de5ca6dad9060f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.563455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf5fd6-06cb-4cd7-b72c-1cf50d328b30 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.564640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1020/2063] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:47 2022] DELETE /identity/v3/users/7303f17b19d345e398de5ca6dad9060f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:47.570133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7367a0-cb7e-4ea5-afe4-d52f1dbba6b1 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.570423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7367a0-cb7e-4ea5-afe4-d52f1dbba6b1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.570918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7367a0-cb7e-4ea5-afe4-d52f1dbba6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.603588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-aa7367a0-cb7e-4ea5-afe4-d52f1dbba6b1 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:47.605010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-aa7367a0-cb7e-4ea5-afe4-d52f1dbba6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.623981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:47.666742 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.667317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.667836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.668400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.669308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:47.669717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.673154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:47.692963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1044/2064] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:30:47.693833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1021/2065] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:47.694688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7367a0-cb7e-4ea5-afe4-d52f1dbba6b1 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:47.697012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:47.698868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.705287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-aa7367a0-cb7e-4ea5-afe4-d52f1dbba6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.715363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117604a9-a1e7-475c-b9e9-1e394ad0e775 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.721951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.722594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.722876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.723205 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] PATH_INFO: `/v3/projects/7661bcddd62d48a8933795df55f5f74f` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.724535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7661bcddd62d48a8933795df55f5f74f)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.724650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7367a0-cb7e-4ea5-afe4-d52f1dbba6b1 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.726611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:47.730458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74e8c5-73b2-4301-8eec-82e95f021210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.731962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74e8c5-73b2-4301-8eec-82e95f021210 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.732870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74e8c5-73b2-4301-8eec-82e95f021210 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.733489 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74e8c5-73b2-4301-8eec-82e95f021210 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.734419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74e8c5-73b2-4301-8eec-82e95f021210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.738682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74e8c5-73b2-4301-8eec-82e95f021210 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:47.775915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1022/2066] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:47 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:47.776809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74e8c5-73b2-4301-8eec-82e95f021210 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:47.781591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3f74e8c5-73b2-4301-8eec-82e95f021210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.816393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7661bcddd62d48a8933795df55f5f74f', 'request_id': 'req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.822401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7661bcddd62d48a8933795df55f5f74f', 'request_id': 'req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.831897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7661bcddd62d48a8933795df55f5f74f', 'request_id': 'req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.853520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] The token cache is being invalidate because project 7661bcddd62d48a8933795df55f5f74f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:47.854040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.855395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:47.858200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1045/2067] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:47 2022] DELETE /identity/v3/projects/7661bcddd62d48a8933795df55f5f74f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:47.859697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:47.861188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.876866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74e8c5-73b2-4301-8eec-82e95f021210 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.931350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79701a1c-86e4-4827-bd50-aae53a3e6d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.931967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79701a1c-86e4-4827-bd50-aae53a3e6d9d service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.932282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79701a1c-86e4-4827-bd50-aae53a3e6d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.932585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79701a1c-86e4-4827-bd50-aae53a3e6d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.934357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc1e87-2b71-4cb7-84b3-c210dd671c3e admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.934502 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-79701a1c-86e4-4827-bd50-aae53a3e6d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.939282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160a4ae-32d5-49be-ac19-88446e77daf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.939820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160a4ae-32d5-49be-ac19-88446e77daf6 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.940095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160a4ae-32d5-49be-ac19-88446e77daf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.940372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160a4ae-32d5-49be-ac19-88446e77daf6 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.941019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160a4ae-32d5-49be-ac19-88446e77daf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.942923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160a4ae-32d5-49be-ac19-88446e77daf6 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:47.947121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79701a1c-86e4-4827-bd50-aae53a3e6d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.948592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79701a1c-86e4-4827-bd50-aae53a3e6d9d service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:47.960065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1023/2068] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:47.961345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e0ea5-8b04-40c5-873f-4e0d8f106115 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.961707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e0ea5-8b04-40c5-873f-4e0d8f106115 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.961985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e0ea5-8b04-40c5-873f-4e0d8f106115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.969030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1046/2069] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:47 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:47.970192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160a4ae-32d5-49be-ac19-88446e77daf6 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:47.984793 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160a4ae-32d5-49be-ac19-88446e77daf6 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:47.989397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-cc8e0ea5-8b04-40c5-873f-4e0d8f106115 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:47.990617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-cc8e0ea5-8b04-40c5-873f-4e0d8f106115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:47.992424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454dc0b-17b4-48c7-90b1-cb14bda9e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:47.993156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454dc0b-17b4-48c7-90b1-cb14bda9e783 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:47.993605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454dc0b-17b4-48c7-90b1-cb14bda9e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:47.993991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454dc0b-17b4-48c7-90b1-cb14bda9e783 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:47.994727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454dc0b-17b4-48c7-90b1-cb14bda9e783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:47.996527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454dc0b-17b4-48c7-90b1-cb14bda9e783 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.000175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1024/2070] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:48.007655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb57da-fd6e-418f-b2ac-947f84c8b860 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.007939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb57da-fd6e-418f-b2ac-947f84c8b860 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.008270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb57da-fd6e-418f-b2ac-947f84c8b860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.012438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1047/2071] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:30:48.013477 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454dc0b-17b4-48c7-90b1-cb14bda9e783 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.026686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454dc0b-17b4-48c7-90b1-cb14bda9e783 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.032813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-4bfb57da-fd6e-418f-b2ac-947f84c8b860 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:48.033842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c0f6f-9ae4-4a53-b043-6ccb5bf4418d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.034341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4bfb57da-fd6e-418f-b2ac-947f84c8b860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.034720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c0f6f-9ae4-4a53-b043-6ccb5bf4418d admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.035379 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c0f6f-9ae4-4a53-b043-6ccb5bf4418d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.035684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c0f6f-9ae4-4a53-b043-6ccb5bf4418d admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.036521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c0f6f-9ae4-4a53-b043-6ccb5bf4418d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.038840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c0f6f-9ae4-4a53-b043-6ccb5bf4418d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.046215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1025/2072] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:48.050477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb57da-fd6e-418f-b2ac-947f84c8b860 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.055141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1048/2073] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:30:48.059273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c0f6f-9ae4-4a53-b043-6ccb5bf4418d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.061285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb57da-fd6e-418f-b2ac-947f84c8b860 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.067602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cbcf4-ccb6-42b8-8e7f-4b6e59860da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.068241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cbcf4-ccb6-42b8-8e7f-4b6e59860da0 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.068536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cbcf4-ccb6-42b8-8e7f-4b6e59860da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.068809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cbcf4-ccb6-42b8-8e7f-4b6e59860da0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.070396 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cbcf4-ccb6-42b8-8e7f-4b6e59860da0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:48.070396 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cbcf4-ccb6-42b8-8e7f-4b6e59860da0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.071639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cbcf4-ccb6-42b8-8e7f-4b6e59860da0 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.074604 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c0f6f-9ae4-4a53-b043-6ccb5bf4418d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.081632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb400f-3707-4fe5-a7b1-0892cdb75920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.082166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:48.082547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb400f-3707-4fe5-a7b1-0892cdb75920 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.082824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb400f-3707-4fe5-a7b1-0892cdb75920 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.083098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb400f-3707-4fe5-a7b1-0892cdb75920 admin admin] PATH_INFO: `/v3/projects/3b0fd49aedc54663a7e97c1e37eb1393/users/d9acbdb683ba4d278753d756ddfa01e6/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.087970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1026/2074] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:30:48.093230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cbcf4-ccb6-42b8-8e7f-4b6e59860da0 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.105113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cbcf4-ccb6-42b8-8e7f-4b6e59860da0 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.105813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb400f-3707-4fe5-a7b1-0892cdb75920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0fd49aedc54663a7e97c1e37eb1393, user_id=d9acbdb683ba4d278753d756ddfa01e6, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.108747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb400f-3707-4fe5-a7b1-0892cdb75920 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.111033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad3f06-9784-4b3f-957d-e7596dcfd4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.111638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad3f06-9784-4b3f-957d-e7596dcfd4fa admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.111922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad3f06-9784-4b3f-957d-e7596dcfd4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.112201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad3f06-9784-4b3f-957d-e7596dcfd4fa admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.112883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad3f06-9784-4b3f-957d-e7596dcfd4fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.115176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad3f06-9784-4b3f-957d-e7596dcfd4fa admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.118940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:48.134899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1049/2075] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:48 2022] PUT /identity/v3/projects/3b0fd49aedc54663a7e97c1e37eb1393/users/d9acbdb683ba4d278753d756ddfa01e6/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:48.147888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57edc7-fa12-4ce7-b7ac-8f8ca718d93f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.148373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57edc7-fa12-4ce7-b7ac-8f8ca718d93f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.148795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57edc7-fa12-4ce7-b7ac-8f8ca718d93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.161935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1027/2076] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:48 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:48.166487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad3f06-9784-4b3f-957d-e7596dcfd4fa admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.183774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ea57edc7-fa12-4ce7-b7ac-8f8ca718d93f None None] MFA Rules not processed for user `d9acbdb683ba4d278753d756ddfa01e6`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:48.185112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ea57edc7-fa12-4ce7-b7ac-8f8ca718d93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.261938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad3f06-9784-4b3f-957d-e7596dcfd4fa admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.271345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98fc0b-7fc3-4953-965f-6e618b8f1e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.271784 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1050/2077] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:48.272834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57edc7-fa12-4ce7-b7ac-8f8ca718d93f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.276576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98fc0b-7fc3-4953-965f-6e618b8f1e61 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.276781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ea57edc7-fa12-4ce7-b7ac-8f8ca718d93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.276904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98fc0b-7fc3-4953-965f-6e618b8f1e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.277133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98fc0b-7fc3-4953-965f-6e618b8f1e61 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.277789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98fc0b-7fc3-4953-965f-6e618b8f1e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.280472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98fc0b-7fc3-4953-965f-6e618b8f1e61 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.292920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57edc7-fa12-4ce7-b7ac-8f8ca718d93f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.298596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.299503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.299615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.299964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] PATH_INFO: `/v3/projects/2c5902fc93984117ba6d3dd8fdc0f6e0` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.301114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5902fc93984117ba6d3dd8fdc0f6e0)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.303041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.307788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1028/2078] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:48 2022] POST /identity/v3/users => generated 285 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:30:48.307788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98fc0b-7fc3-4953-965f-6e618b8f1e61 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.307788 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ce98fc0b-7fc3-4953-965f-6e618b8f1e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.365578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5902fc93984117ba6d3dd8fdc0f6e0', 'request_id': 'req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.367935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98fc0b-7fc3-4953-965f-6e618b8f1e61 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.371290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5902fc93984117ba6d3dd8fdc0f6e0', 'request_id': 'req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.378485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5902fc93984117ba6d3dd8fdc0f6e0', 'request_id': 'req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.391150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] The token cache is being invalidate because project 2c5902fc93984117ba6d3dd8fdc0f6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:48.391436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.392200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.393736 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1051/2079] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:48 2022] DELETE /identity/v3/projects/2c5902fc93984117ba6d3dd8fdc0f6e0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:48.397343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.398721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.418618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7b5f3-610f-41fe-81f2-edc1c61b09e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.419233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7b5f3-610f-41fe-81f2-edc1c61b09e5 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.419510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7b5f3-610f-41fe-81f2-edc1c61b09e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.420294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7b5f3-610f-41fe-81f2-edc1c61b09e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.422265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9fe7b5f3-610f-41fe-81f2-edc1c61b09e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.462006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:48.467049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325cd7a-fa4f-4fbc-b82e-f3c9f4f1441f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.471924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2191b81-1dfa-463b-9b02-e8f387f16d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.472545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2191b81-1dfa-463b-9b02-e8f387f16d04 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.472827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2191b81-1dfa-463b-9b02-e8f387f16d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.473100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2191b81-1dfa-463b-9b02-e8f387f16d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.473739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2191b81-1dfa-463b-9b02-e8f387f16d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.475333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2191b81-1dfa-463b-9b02-e8f387f16d04 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.483007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7b5f3-610f-41fe-81f2-edc1c61b09e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.484563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7b5f3-610f-41fe-81f2-edc1c61b09e5 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.486034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1052/2080] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:48.486858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2191b81-1dfa-463b-9b02-e8f387f16d04 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.488625 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b2191b81-1dfa-463b-9b02-e8f387f16d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.495017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1029/2081] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:48.495834 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7b5f3-610f-41fe-81f2-edc1c61b09e5 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.501765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2191b81-1dfa-463b-9b02-e8f387f16d04 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.506360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7b5f3-610f-41fe-81f2-edc1c61b09e5 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.507225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.507857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.508161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.508413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] PATH_INFO: `/v3/users/f6396078f04a40eab512463bece02d50` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.509594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6396078f04a40eab512463bece02d50)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.511241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed94511-0614-472b-8e3e-fdd47be7dc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.511721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.511819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed94511-0614-472b-8e3e-fdd47be7dc5e admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.512102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed94511-0614-472b-8e3e-fdd47be7dc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.514256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:48.516621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed94511-0614-472b-8e3e-fdd47be7dc5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.517222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed94511-0614-472b-8e3e-fdd47be7dc5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.518914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed94511-0614-472b-8e3e-fdd47be7dc5e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.529887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1030/2082] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:48.530716 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed94511-0614-472b-8e3e-fdd47be7dc5e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.532382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fed94511-0614-472b-8e3e-fdd47be7dc5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.545202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed94511-0614-472b-8e3e-fdd47be7dc5e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.550207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.550952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.551259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.551707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] PATH_INFO: `/v3/users/d9acbdb683ba4d278753d756ddfa01e6` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.552984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9acbdb683ba4d278753d756ddfa01e6)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.554905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.567511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6396078f04a40eab512463bece02d50'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.568247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6396078f04a40eab512463bece02d50'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.575954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6396078f04a40eab512463bece02d50'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.585965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6396078f04a40eab512463bece02d50'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.586746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6396078f04a40eab512463bece02d50'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.610373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1053/2083] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:48 2022] DELETE /identity/v3/users/f6396078f04a40eab512463bece02d50 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:48.610373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.610373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.614236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9acbdb683ba4d278753d756ddfa01e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.615259 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9acbdb683ba4d278753d756ddfa01e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.648385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9acbdb683ba4d278753d756ddfa01e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.662385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9acbdb683ba4d278753d756ddfa01e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.676714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ad25fdfb-3c68-4d54-a97c-9bd403c1f8ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9acbdb683ba4d278753d756ddfa01e6'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:48.689999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1031/2084] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:48 2022] DELETE /identity/v3/users/d9acbdb683ba4d278753d756ddfa01e6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:48.695864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde598f-be00-4285-98e5-d12242946214 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.696450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde598f-be00-4285-98e5-d12242946214 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.696951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde598f-be00-4285-98e5-d12242946214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.715956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c8e74-485c-4ab9-b7c1-b9963283a255 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:48.727585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-5dde598f-be00-4285-98e5-d12242946214 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:48.729329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5dde598f-be00-4285-98e5-d12242946214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.791179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1ac97-180f-4d8d-955b-7eabe4e83887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:48.792523 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1ac97-180f-4d8d-955b-7eabe4e83887 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.792896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1ac97-180f-4d8d-955b-7eabe4e83887 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.793239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1ac97-180f-4d8d-955b-7eabe4e83887 admin admin] PATH_INFO: `/v3/projects/bd12f59f78604ba8adbedb120ad71043/users/caf373f5e5c8414c8398c9d2be128093/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.821169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1ac97-180f-4d8d-955b-7eabe4e83887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd12f59f78604ba8adbedb120ad71043, user_id=caf373f5e5c8414c8398c9d2be128093, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:48.824533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1ac97-180f-4d8d-955b-7eabe4e83887 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:48.846328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1032/2085] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:48.859196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde598f-be00-4285-98e5-d12242946214 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:48.860919 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5dde598f-be00-4285-98e5-d12242946214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.867450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1054/2086] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:48 2022] PUT /identity/v3/projects/bd12f59f78604ba8adbedb120ad71043/users/caf373f5e5c8414c8398c9d2be128093/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:48.881159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216ebe-6497-4093-9db4-80a7bf8da34f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:48.881580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216ebe-6497-4093-9db4-80a7bf8da34f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:48.881955 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216ebe-6497-4093-9db4-80a7bf8da34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:48.904990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:48.922738 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3e216ebe-6497-4093-9db4-80a7bf8da34f None None] MFA Rules not processed for user `caf373f5e5c8414c8398c9d2be128093`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:48.924812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3e216ebe-6497-4093-9db4-80a7bf8da34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:48.950239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:49.005755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde598f-be00-4285-98e5-d12242946214 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:49.007245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1055/2087] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:49.008171 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e216ebe-6497-4093-9db4-80a7bf8da34f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:49.009661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3e216ebe-6497-4093-9db4-80a7bf8da34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.078805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2a465-dec3-4206-b4e8-188c2172a974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:49.079423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2a465-dec3-4206-b4e8-188c2172a974 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.079704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2a465-dec3-4206-b4e8-188c2172a974 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.079975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2a465-dec3-4206-b4e8-188c2172a974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.112424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e216ebe-6497-4093-9db4-80a7bf8da34f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:49.117375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:49.118111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.118509 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.118883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] PATH_INFO: `/v3/projects/cae782b9d24842f9a940ffbabf017e97` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.120123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae782b9d24842f9a940ffbabf017e97)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:49.122195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:49.150481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2a465-dec3-4206-b4e8-188c2172a974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:49.151944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2a465-dec3-4206-b4e8-188c2172a974 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:49.162769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1033/2088] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:49.163631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2a465-dec3-4206-b4e8-188c2172a974 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:49.165093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9be2a465-dec3-4206-b4e8-188c2172a974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.173614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae782b9d24842f9a940ffbabf017e97', 'request_id': 'req-56016753-9076-4004-93fe-36db8f980a23'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.181529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae782b9d24842f9a940ffbabf017e97', 'request_id': 'req-56016753-9076-4004-93fe-36db8f980a23'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.191879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae782b9d24842f9a940ffbabf017e97', 'request_id': 'req-56016753-9076-4004-93fe-36db8f980a23'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.211981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] The token cache is being invalidate because project cae782b9d24842f9a940ffbabf017e97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:49.212672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.213536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.215516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1056/2089] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:49 2022] DELETE /identity/v3/projects/cae782b9d24842f9a940ffbabf017e97 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:49.220846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:49.222457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.236332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2a465-dec3-4206-b4e8-188c2172a974 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:49.273954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:49.310221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:49.311188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.311633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.312485 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.315657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.317122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56016753-9076-4004-93fe-36db8f980a23 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:49.322750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:49.323668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.324014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.324353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] PATH_INFO: `/v3/users/2303f1922c27443fa977f61c2ebe2c16` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.329518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2303f1922c27443fa977f61c2ebe2c16)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:49.331704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:49.334831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:49.387463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2303f1922c27443fa977f61c2ebe2c16'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.388730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2303f1922c27443fa977f61c2ebe2c16'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.397189 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:49.397765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2303f1922c27443fa977f61c2ebe2c16'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.398700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:49.406253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2303f1922c27443fa977f61c2ebe2c16'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.406952 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2303f1922c27443fa977f61c2ebe2c16'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.409721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1034/2090] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:49.431796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:49.433298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.438457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1057/2091] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:49 2022] DELETE /identity/v3/users/2303f1922c27443fa977f61c2ebe2c16 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:49.448218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d627a-c2ea-4f76-81e1-7e5496c39296 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:49.495556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:49.497533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.517174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:49.517818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.518094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.518366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] PATH_INFO: `/v3/users/caf373f5e5c8414c8398c9d2be128093` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.519711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf373f5e5c8414c8398c9d2be128093)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:49.521915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:49.580740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf373f5e5c8414c8398c9d2be128093'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.581499 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf373f5e5c8414c8398c9d2be128093'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.595699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf373f5e5c8414c8398c9d2be128093'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.603646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf373f5e5c8414c8398c9d2be128093'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.614923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-842454bd-06fe-4666-8b69-19568189e45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf373f5e5c8414c8398c9d2be128093'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.620035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1035/2092] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:49 2022] DELETE /identity/v3/users/caf373f5e5c8414c8398c9d2be128093 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:49.624886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd886e7-735d-4bb9-b993-99d25c1c7fc1 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.625344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd886e7-735d-4bb9-b993-99d25c1c7fc1 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.625843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd886e7-735d-4bb9-b993-99d25c1c7fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.630297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817e1bd-e635-4833-9457-8f132496d180 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:49.648594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-edd886e7-735d-4bb9-b993-99d25c1c7fc1 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:49.650353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-edd886e7-735d-4bb9-b993-99d25c1c7fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.690895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:49.696700 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:49.712587 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:49.713227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.713505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.713776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] PATH_INFO: `/v3/projects/3b0fd49aedc54663a7e97c1e37eb1393` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.715148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0fd49aedc54663a7e97c1e37eb1393)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:49.717652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:49.728528 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1036/2093] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:49.739734 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd886e7-735d-4bb9-b993-99d25c1c7fc1 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:49.741330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-edd886e7-735d-4bb9-b993-99d25c1c7fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.784489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0fd49aedc54663a7e97c1e37eb1393', 'request_id': 'req-42f34542-8232-4b64-931f-37b46e31ee0d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.794427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0fd49aedc54663a7e97c1e37eb1393', 'request_id': 'req-42f34542-8232-4b64-931f-37b46e31ee0d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.801704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0fd49aedc54663a7e97c1e37eb1393', 'request_id': 'req-42f34542-8232-4b64-931f-37b46e31ee0d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.810007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd886e7-735d-4bb9-b993-99d25c1c7fc1 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:49.815078 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] The token cache is being invalidate because project 3b0fd49aedc54663a7e97c1e37eb1393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:49.815369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.816094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.817616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1058/2094] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:49 2022] DELETE /identity/v3/projects/3b0fd49aedc54663a7e97c1e37eb1393 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:49.818384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:49.819709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.865258 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c6e69-8a89-4432-b0ac-39c0942b02ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:49.865860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c6e69-8a89-4432-b0ac-39c0942b02ae service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.866214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c6e69-8a89-4432-b0ac-39c0942b02ae service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.866489 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c6e69-8a89-4432-b0ac-39c0942b02ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.868373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-956c6e69-8a89-4432-b0ac-39c0942b02ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.884973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34542-8232-4b64-931f-37b46e31ee0d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:49.890303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:49.890927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.891201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.891469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] PATH_INFO: `/v3/projects/3a9ebf303399472d8b1aa1b420c436b4` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.892602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9ebf303399472d8b1aa1b420c436b4)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:49.894852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:49.942690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c6e69-8a89-4432-b0ac-39c0942b02ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:49.944148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c6e69-8a89-4432-b0ac-39c0942b02ae service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:49.948363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9ebf303399472d8b1aa1b420c436b4', 'request_id': 'req-9606fe39-6cc5-4631-a863-12db67a73c35'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.956948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9ebf303399472d8b1aa1b420c436b4', 'request_id': 'req-9606fe39-6cc5-4631-a863-12db67a73c35'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.959195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1037/2095] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:49.961476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca9087-c59a-4c24-91cd-ef648ff71b3e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:49.961599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca9087-c59a-4c24-91cd-ef648ff71b3e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:49.961975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca9087-c59a-4c24-91cd-ef648ff71b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:49.968425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9ebf303399472d8b1aa1b420c436b4', 'request_id': 'req-9606fe39-6cc5-4631-a863-12db67a73c35'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.987925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] The token cache is being invalidate because project 3a9ebf303399472d8b1aa1b420c436b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:49.988252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.988903 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:49.990120 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-deca9087-c59a-4c24-91cd-ef648ff71b3e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:49.991337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-deca9087-c59a-4c24-91cd-ef648ff71b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:49.996593 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1059/2096] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:49 2022] DELETE /identity/v3/projects/3a9ebf303399472d8b1aa1b420c436b4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:49.998349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:49.999896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.028075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:50.047058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:50.072431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1038/2097] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:50.074080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573c4e9-7d80-41f1-a0e2-96d74ca4699f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.074471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573c4e9-7d80-41f1-a0e2-96d74ca4699f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.074816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573c4e9-7d80-41f1-a0e2-96d74ca4699f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.087375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606fe39-6cc5-4631-a863-12db67a73c35 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.092241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.092905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.093239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.093513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] PATH_INFO: `/v3/users/e1dc9c62ddd44b53a51d625c5b81331d` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.094646 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1dc9c62ddd44b53a51d625c5b81331d)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.096642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.096798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-8573c4e9-7d80-41f1-a0e2-96d74ca4699f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:50.098169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8573c4e9-7d80-41f1-a0e2-96d74ca4699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.109585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1039/2098] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:50.110787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf4d73-c86f-4a13-8a7d-2b7b095eb206 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.111065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf4d73-c86f-4a13-8a7d-2b7b095eb206 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.111335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf4d73-c86f-4a13-8a7d-2b7b095eb206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.145246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b0cf4d73-c86f-4a13-8a7d-2b7b095eb206 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:50.151319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b0cf4d73-c86f-4a13-8a7d-2b7b095eb206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.174191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1dc9c62ddd44b53a51d625c5b81331d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.175193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1dc9c62ddd44b53a51d625c5b81331d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.175743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1040/2099] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:50.177139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd6184-2be2-4690-9ecb-af8e0be15733 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.177428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd6184-2be2-4690-9ecb-af8e0be15733 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.177756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd6184-2be2-4690-9ecb-af8e0be15733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.190963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1dc9c62ddd44b53a51d625c5b81331d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.199933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1dc9c62ddd44b53a51d625c5b81331d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.200779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1dc9c62ddd44b53a51d625c5b81331d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.214056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b6dd6184-2be2-4690-9ecb-af8e0be15733 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:50.217228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b6dd6184-2be2-4690-9ecb-af8e0be15733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.230033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1041/2100] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:50.230957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd6184-2be2-4690-9ecb-af8e0be15733 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.232367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b6dd6184-2be2-4690-9ecb-af8e0be15733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.240009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1060/2101] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:49 2022] DELETE /identity/v3/users/e1dc9c62ddd44b53a51d625c5b81331d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:50.242370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.248556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd6184-2be2-4690-9ecb-af8e0be15733 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.315185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30766f07-81fd-44cd-bb49-5e588b881324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.319051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30766f07-81fd-44cd-bb49-5e588b881324 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.319339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30766f07-81fd-44cd-bb49-5e588b881324 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.319606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30766f07-81fd-44cd-bb49-5e588b881324 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.320454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30766f07-81fd-44cd-bb49-5e588b881324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:50.320781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30766f07-81fd-44cd-bb49-5e588b881324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.322446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30766f07-81fd-44cd-bb49-5e588b881324 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.327951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a080c05-d237-4167-8060-e74342350f8d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.334428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe3bc3-34db-4563-97e0-dc4d0aa50ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.335057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe3bc3-34db-4563-97e0-dc4d0aa50ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.335334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe3bc3-34db-4563-97e0-dc4d0aa50ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.335608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe3bc3-34db-4563-97e0-dc4d0aa50ff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.336309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe3bc3-34db-4563-97e0-dc4d0aa50ff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:50.336648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe3bc3-34db-4563-97e0-dc4d0aa50ff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.338339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe3bc3-34db-4563-97e0-dc4d0aa50ff9 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.339634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1042/2102] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:50.342081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e9d37-8cf1-4ac6-a137-144a1adca51d None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.342740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e9d37-8cf1-4ac6-a137-144a1adca51d None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.343344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e9d37-8cf1-4ac6-a137-144a1adca51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.359456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1061/2103] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:50.365624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe3bc3-34db-4563-97e0-dc4d0aa50ff9 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.375128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-021e9d37-8cf1-4ac6-a137-144a1adca51d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:50.379163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-021e9d37-8cf1-4ac6-a137-144a1adca51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.390860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe3bc3-34db-4563-97e0-dc4d0aa50ff9 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.392128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1043/2104] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:50.393412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e9d37-8cf1-4ac6-a137-144a1adca51d None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.403646 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532a804-613b-49dd-9821-b6183f418c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.404814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532a804-613b-49dd-9821-b6183f418c0f admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.405091 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532a804-613b-49dd-9821-b6183f418c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.405360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532a804-613b-49dd-9821-b6183f418c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.405999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532a804-613b-49dd-9821-b6183f418c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.407984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532a804-613b-49dd-9821-b6183f418c0f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.415108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e9d37-8cf1-4ac6-a137-144a1adca51d None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.420848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55bebe-68f1-49fc-acdf-72761138315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.421469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55bebe-68f1-49fc-acdf-72761138315f admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.421767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55bebe-68f1-49fc-acdf-72761138315f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.422010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55bebe-68f1-49fc-acdf-72761138315f admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.422635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55bebe-68f1-49fc-acdf-72761138315f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.424559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55bebe-68f1-49fc-acdf-72761138315f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.453213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1044/2105] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:50.456782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1062/2106] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:50.456930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55bebe-68f1-49fc-acdf-72761138315f admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.457025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532a804-613b-49dd-9821-b6183f418c0f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.457025 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e532a804-613b-49dd-9821-b6183f418c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.458548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4a55bebe-68f1-49fc-acdf-72761138315f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.467655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:50.518624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55bebe-68f1-49fc-acdf-72761138315f admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.530401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532a804-613b-49dd-9821-b6183f418c0f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.535299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.535931 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.536206 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.536476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] PATH_INFO: `/v3/projects/bd12f59f78604ba8adbedb120ad71043` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.537789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd12f59f78604ba8adbedb120ad71043)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.539759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.566440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:50.579561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae96f09-5207-40ba-b035-c2086949d7b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.580221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae96f09-5207-40ba-b035-c2086949d7b6 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.580523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae96f09-5207-40ba-b035-c2086949d7b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.580791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae96f09-5207-40ba-b035-c2086949d7b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.592370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd12f59f78604ba8adbedb120ad71043', 'request_id': 'req-fff6eef2-90c2-41b6-bac7-810b9047770b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.593625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae96f09-5207-40ba-b035-c2086949d7b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.595967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae96f09-5207-40ba-b035-c2086949d7b6 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.598842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd12f59f78604ba8adbedb120ad71043', 'request_id': 'req-fff6eef2-90c2-41b6-bac7-810b9047770b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.605134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd12f59f78604ba8adbedb120ad71043', 'request_id': 'req-fff6eef2-90c2-41b6-bac7-810b9047770b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.606333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1045/2107] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:50.606948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae96f09-5207-40ba-b035-c2086949d7b6 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.617068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae96f09-5207-40ba-b035-c2086949d7b6 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.634581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] The token cache is being invalidate because project bd12f59f78604ba8adbedb120ad71043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:50.634908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.636991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:50.640773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1063/2108] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:50 2022] DELETE /identity/v3/projects/bd12f59f78604ba8adbedb120ad71043 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:50.642341 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.644443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.715434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1944a0f-6617-43ee-b203-ce5d4b9297b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.716207 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1944a0f-6617-43ee-b203-ce5d4b9297b1 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.716783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1944a0f-6617-43ee-b203-ce5d4b9297b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.717127 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1944a0f-6617-43ee-b203-ce5d4b9297b1 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.717872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1944a0f-6617-43ee-b203-ce5d4b9297b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.720083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1944a0f-6617-43ee-b203-ce5d4b9297b1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.731676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6eef2-90c2-41b6-bac7-810b9047770b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.742158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52b94b-7fd0-485f-97d9-7a540bda8b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.744987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52b94b-7fd0-485f-97d9-7a540bda8b1d admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.745704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52b94b-7fd0-485f-97d9-7a540bda8b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.746489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52b94b-7fd0-485f-97d9-7a540bda8b1d admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.747941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52b94b-7fd0-485f-97d9-7a540bda8b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.749440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1046/2109] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:50.751004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46353cd1-f0a3-4bfe-9b51-e28f89c3b559 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.751292 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46353cd1-f0a3-4bfe-9b51-e28f89c3b559 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.751467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52b94b-7fd0-485f-97d9-7a540bda8b1d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.751560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46353cd1-f0a3-4bfe-9b51-e28f89c3b559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.774020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-46353cd1-f0a3-4bfe-9b51-e28f89c3b559 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:50.776104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-46353cd1-f0a3-4bfe-9b51-e28f89c3b559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.785435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1064/2110] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:50.786424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52b94b-7fd0-485f-97d9-7a540bda8b1d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.788114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1f52b94b-7fd0-485f-97d9-7a540bda8b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.794244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1047/2111] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:50.795677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46353cd1-f0a3-4bfe-9b51-e28f89c3b559 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.810547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46353cd1-f0a3-4bfe-9b51-e28f89c3b559 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.811102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52b94b-7fd0-485f-97d9-7a540bda8b1d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.817150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792261a-9bcd-415e-9a2e-7d6dc91710e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.818712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792261a-9bcd-415e-9a2e-7d6dc91710e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.819546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ffad3-1229-467d-ad86-418529f63762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.820366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ffad3-1229-467d-ad86-418529f63762 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.820675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ffad3-1229-467d-ad86-418529f63762 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.820946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ffad3-1229-467d-ad86-418529f63762 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.821599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ffad3-1229-467d-ad86-418529f63762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.823310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ffad3-1229-467d-ad86-418529f63762 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.824922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792261a-9bcd-415e-9a2e-7d6dc91710e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.825363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792261a-9bcd-415e-9a2e-7d6dc91710e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.826825 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792261a-9bcd-415e-9a2e-7d6dc91710e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.828665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c792261a-9bcd-415e-9a2e-7d6dc91710e9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.837192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1065/2112] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:50.841617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8be367-01d2-45d7-a74b-68ef8d5063bf None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.842001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8be367-01d2-45d7-a74b-68ef8d5063bf None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.842400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8be367-01d2-45d7-a74b-68ef8d5063bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.845994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1048/2113] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:50.847097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792261a-9bcd-415e-9a2e-7d6dc91710e9 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.859783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792261a-9bcd-415e-9a2e-7d6dc91710e9 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.863576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-4a8be367-01d2-45d7-a74b-68ef8d5063bf None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:50.864829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4a8be367-01d2-45d7-a74b-68ef8d5063bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:50.865099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23b6f5-776a-4359-a5e6-f58d4ecb2c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.865805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23b6f5-776a-4359-a5e6-f58d4ecb2c37 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.866094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23b6f5-776a-4359-a5e6-f58d4ecb2c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.866602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23b6f5-776a-4359-a5e6-f58d4ecb2c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.867007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23b6f5-776a-4359-a5e6-f58d4ecb2c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.868972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23b6f5-776a-4359-a5e6-f58d4ecb2c37 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.877141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1066/2114] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:50.877141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8be367-01d2-45d7-a74b-68ef8d5063bf None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.881208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1049/2115] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:50.882203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23b6f5-776a-4359-a5e6-f58d4ecb2c37 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.887769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8be367-01d2-45d7-a74b-68ef8d5063bf None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.893058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899e840-22cf-485d-8245-0668f1d62b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.893674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899e840-22cf-485d-8245-0668f1d62b7c admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.893813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23b6f5-776a-4359-a5e6-f58d4ecb2c37 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.893953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899e840-22cf-485d-8245-0668f1d62b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.894228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899e840-22cf-485d-8245-0668f1d62b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.895368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899e840-22cf-485d-8245-0668f1d62b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.896482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899e840-22cf-485d-8245-0668f1d62b7c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.898759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209be41-5ad8-4a06-a795-e16fcdbc3810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.899079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:50.899362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209be41-5ad8-4a06-a795-e16fcdbc3810 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.899646 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209be41-5ad8-4a06-a795-e16fcdbc3810 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.900034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209be41-5ad8-4a06-a795-e16fcdbc3810 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.900732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209be41-5ad8-4a06-a795-e16fcdbc3810 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:50.901053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209be41-5ad8-4a06-a795-e16fcdbc3810 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.902638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209be41-5ad8-4a06-a795-e16fcdbc3810 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.911929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1067/2116] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:50.912902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899e840-22cf-485d-8245-0668f1d62b7c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.918214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1050/2117] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:50.919036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209be41-5ad8-4a06-a795-e16fcdbc3810 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.930724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209be41-5ad8-4a06-a795-e16fcdbc3810 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.931974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899e840-22cf-485d-8245-0668f1d62b7c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:50.937055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34611aad-0cec-4ee2-8c83-6f9abd6ea2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.937808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34611aad-0cec-4ee2-8c83-6f9abd6ea2af admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.937922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc982-36b0-4721-aa5c-3029d5683966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:50.938088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34611aad-0cec-4ee2-8c83-6f9abd6ea2af admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.938363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34611aad-0cec-4ee2-8c83-6f9abd6ea2af admin admin] PATH_INFO: `/v3/projects/13ec47a00b094822929aa10ecff3693d/users/27553f7a13e54b0c8758ca64d6229b78/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.938499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc982-36b0-4721-aa5c-3029d5683966 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:50.938744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc982-36b0-4721-aa5c-3029d5683966 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:50.939038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc982-36b0-4721-aa5c-3029d5683966 admin admin] PATH_INFO: `/v3/projects/32e68213ef7041a79aaaf1d255d2395e/users/b86acd50276842a3bc18e3ffa00e515a/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:50.955355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fc982-36b0-4721-aa5c-3029d5683966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e68213ef7041a79aaaf1d255d2395e, user_id=b86acd50276842a3bc18e3ffa00e515a, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.956240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34611aad-0cec-4ee2-8c83-6f9abd6ea2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ec47a00b094822929aa10ecff3693d, user_id=27553f7a13e54b0c8758ca64d6229b78, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:50.958365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58fc982-36b0-4721-aa5c-3029d5683966 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.964697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34611aad-0cec-4ee2-8c83-6f9abd6ea2af admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:50.974496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1068/2118] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:50 2022] PUT /identity/v3/projects/32e68213ef7041a79aaaf1d255d2395e/users/b86acd50276842a3bc18e3ffa00e515a/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:50.979211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc982-36b0-4721-aa5c-3029d5683966 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.979436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1051/2119] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:50 2022] PUT /identity/v3/projects/13ec47a00b094822929aa10ecff3693d/users/27553f7a13e54b0c8758ca64d6229b78/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:50.980438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34611aad-0cec-4ee2-8c83-6f9abd6ea2af admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:50.981949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-34611aad-0cec-4ee2-8c83-6f9abd6ea2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.057277 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc982-36b0-4721-aa5c-3029d5683966 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:51.075728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34611aad-0cec-4ee2-8c83-6f9abd6ea2af admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:51.080697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:51.081327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.081608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.081877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] PATH_INFO: `/v3/projects/76b9ccd09b574daba079f9e610970939` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.083258 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b9ccd09b574daba079f9e610970939)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:51.085221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:51.097822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:51.127956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3752273-6247-41c9-bbfc-274e65f2fb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:51.128677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3752273-6247-41c9-bbfc-274e65f2fb4f admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.129027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3752273-6247-41c9-bbfc-274e65f2fb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.129363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3752273-6247-41c9-bbfc-274e65f2fb4f admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.130073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3752273-6247-41c9-bbfc-274e65f2fb4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:51.131987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3752273-6247-41c9-bbfc-274e65f2fb4f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:51.144573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b9ccd09b574daba079f9e610970939', 'request_id': 'req-5fd64406-9309-4cdf-9a3e-d5d9a584da57'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.150609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b9ccd09b574daba079f9e610970939', 'request_id': 'req-5fd64406-9309-4cdf-9a3e-d5d9a584da57'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.157645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b9ccd09b574daba079f9e610970939', 'request_id': 'req-5fd64406-9309-4cdf-9a3e-d5d9a584da57'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.166319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1069/2120] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:50 2022] POST /identity/v3/projects => generated 363 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:51.166319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574ecf0-29c9-4bad-ad1e-09168c746bfb None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.166319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574ecf0-29c9-4bad-ad1e-09168c746bfb None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.166319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574ecf0-29c9-4bad-ad1e-09168c746bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.176709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] The token cache is being invalidate because project 76b9ccd09b574daba079f9e610970939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:51.177226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.177846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5fd64406-9309-4cdf-9a3e-d5d9a584da57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.179718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1052/2121] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:50 2022] DELETE /identity/v3/projects/76b9ccd09b574daba079f9e610970939 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:51.180841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68772a41-0dba-47b3-ac0f-53159b9ffe74 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.181193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68772a41-0dba-47b3-ac0f-53159b9ffe74 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.181548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68772a41-0dba-47b3-ac0f-53159b9ffe74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.189122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f574ecf0-29c9-4bad-ad1e-09168c746bfb None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:51.190338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f574ecf0-29c9-4bad-ad1e-09168c746bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.213865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-68772a41-0dba-47b3-ac0f-53159b9ffe74 None None] MFA Rules not processed for user `27553f7a13e54b0c8758ca64d6229b78`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:51.215185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-68772a41-0dba-47b3-ac0f-53159b9ffe74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.276235 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1070/2122] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:51.277166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574ecf0-29c9-4bad-ad1e-09168c746bfb None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:51.278507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f574ecf0-29c9-4bad-ad1e-09168c746bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.293574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574ecf0-29c9-4bad-ad1e-09168c746bfb None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:51.301357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797419a5-b04c-4a5d-a212-9c4482ac815b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:51.302400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797419a5-b04c-4a5d-a212-9c4482ac815b admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.303131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797419a5-b04c-4a5d-a212-9c4482ac815b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.303264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797419a5-b04c-4a5d-a212-9c4482ac815b admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.303921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797419a5-b04c-4a5d-a212-9c4482ac815b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:51.305976 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797419a5-b04c-4a5d-a212-9c4482ac815b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:51.314454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1053/2123] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:51.314454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68772a41-0dba-47b3-ac0f-53159b9ffe74 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:51.314454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-68772a41-0dba-47b3-ac0f-53159b9ffe74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.321899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:51.330578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68772a41-0dba-47b3-ac0f-53159b9ffe74 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:51.336173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:51.336840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.337136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.337394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] PATH_INFO: `/v3/users/c490084fe6e74cdbaa024b0bb465df9c` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.338519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c490084fe6e74cdbaa024b0bb465df9c)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:51.339961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1071/2124] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:51 2022] POST /identity/v3/users => generated 282 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:51.341277 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:51.341765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797419a5-b04c-4a5d-a212-9c4482ac815b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:51.344161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-797419a5-b04c-4a5d-a212-9c4482ac815b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.409542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c490084fe6e74cdbaa024b0bb465df9c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.410328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c490084fe6e74cdbaa024b0bb465df9c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.424821 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c490084fe6e74cdbaa024b0bb465df9c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.434267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c490084fe6e74cdbaa024b0bb465df9c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.442876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797419a5-b04c-4a5d-a212-9c4482ac815b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:51.445814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c490084fe6e74cdbaa024b0bb465df9c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:51.454075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:51.456541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.456805 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.457088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.457949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1054/2125] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:51 2022] DELETE /identity/v3/users/c490084fe6e74cdbaa024b0bb465df9c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:51.458922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:51.460903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.462704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.469857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:51.481542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:51.485284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:51.508682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1072/2126] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:51.509699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:51.511420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.528843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a02bc6-b25a-405c-ba84-95f85758234b service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:51.597799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e442b6-6d1d-4f14-a5f1-0f62928dd283 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:51.602880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c69ec-00c8-4f98-9871-8dd26aea8e77 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52470b620abd4cd29ab7e4752f285bb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:51.603778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c69ec-00c8-4f98-9871-8dd26aea8e77 service glance] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.604170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c69ec-00c8-4f98-9871-8dd26aea8e77 service glance] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.604570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c69ec-00c8-4f98-9871-8dd26aea8e77 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.606284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628981e-f238-4443-92dd-6dcf8814fa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:51.607062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628981e-f238-4443-92dd-6dcf8814fa8e admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.607882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2a1c69ec-00c8-4f98-9871-8dd26aea8e77 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.608372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628981e-f238-4443-92dd-6dcf8814fa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.608820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628981e-f238-4443-92dd-6dcf8814fa8e admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.609791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628981e-f238-4443-92dd-6dcf8814fa8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:51.611623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628981e-f238-4443-92dd-6dcf8814fa8e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:51.624117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1073/2127] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:51.628526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628981e-f238-4443-92dd-6dcf8814fa8e admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:51.639266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628981e-f238-4443-92dd-6dcf8814fa8e admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:51.644509 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa1111-76a2-43db-b5f5-6118512fb8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:51.645152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa1111-76a2-43db-b5f5-6118512fb8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.645421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa1111-76a2-43db-b5f5-6118512fb8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.645728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa1111-76a2-43db-b5f5-6118512fb8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.646462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa1111-76a2-43db-b5f5-6118512fb8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:51.648152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa1111-76a2-43db-b5f5-6118512fb8d4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:51.665066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1074/2128] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:51.666218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa1111-76a2-43db-b5f5-6118512fb8d4 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:51.676573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa1111-76a2-43db-b5f5-6118512fb8d4 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:51.681578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734a21e-3203-43ad-b5c9-567f304dd5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:51.682243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a21e-3203-43ad-b5c9-567f304dd5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.682510 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a21e-3203-43ad-b5c9-567f304dd5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.682781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a21e-3203-43ad-b5c9-567f304dd5cf admin admin] PATH_INFO: `/v3/projects/38d02d083ec24587add6fa8c2ae389a5/users/894bb815eb0d424aa01ebc941a2349c6/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.690337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c69ec-00c8-4f98-9871-8dd26aea8e77 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:51.692050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c69ec-00c8-4f98-9871-8dd26aea8e77 service glance] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:51.700199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734a21e-3203-43ad-b5c9-567f304dd5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d02d083ec24587add6fa8c2ae389a5, user_id=894bb815eb0d424aa01ebc941a2349c6, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:51.703968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734a21e-3203-43ad-b5c9-567f304dd5cf admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:51.707126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1055/2129] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:51.726248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1075/2130] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:51 2022] PUT /identity/v3/projects/38d02d083ec24587add6fa8c2ae389a5/users/894bb815eb0d424aa01ebc941a2349c6/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:51.739392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f55e07-6c28-4432-87d4-603546850d4a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:51.739682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f55e07-6c28-4432-87d4-603546850d4a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:51.740103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f55e07-6c28-4432-87d4-603546850d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:51.785720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-f3f55e07-6c28-4432-87d4-603546850d4a None None] MFA Rules not processed for user `894bb815eb0d424aa01ebc941a2349c6`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:51.787350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f3f55e07-6c28-4432-87d4-603546850d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.932585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1056/2131] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:51.936162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734a21e-3203-43ad-b5c9-567f304dd5cf admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:51.947444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2734a21e-3203-43ad-b5c9-567f304dd5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:51.984291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f55e07-6c28-4432-87d4-603546850d4a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:51.986946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f3f55e07-6c28-4432-87d4-603546850d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:52.049583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:52.054727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734a21e-3203-43ad-b5c9-567f304dd5cf admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:52.059504 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80425bd-228b-4798-9d95-720fec974601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:52.061438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80425bd-228b-4798-9d95-720fec974601 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.061884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80425bd-228b-4798-9d95-720fec974601 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.062361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80425bd-228b-4798-9d95-720fec974601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.092433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:52.098330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80425bd-228b-4798-9d95-720fec974601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:52.099962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80425bd-228b-4798-9d95-720fec974601 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:52.113516 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f55e07-6c28-4432-87d4-603546850d4a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:52.120933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:52.121198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1076/2132] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:30:52.121998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.122284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.122563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] PATH_INFO: `/v3/projects/db93bdd05f964880a0d7f84977bc199d` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.124697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db93bdd05f964880a0d7f84977bc199d)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:52.126708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:52.208104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db93bdd05f964880a0d7f84977bc199d', 'request_id': 'req-54688a3d-e88e-4184-9a18-6d45ab146d33'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:52.220186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db93bdd05f964880a0d7f84977bc199d', 'request_id': 'req-54688a3d-e88e-4184-9a18-6d45ab146d33'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:52.228145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db93bdd05f964880a0d7f84977bc199d', 'request_id': 'req-54688a3d-e88e-4184-9a18-6d45ab146d33'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:52.254988 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] The token cache is being invalidate because project db93bdd05f964880a0d7f84977bc199d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:52.255741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:52.256906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-54688a3d-e88e-4184-9a18-6d45ab146d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:52.260759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1057/2133] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:51 2022] DELETE /identity/v3/projects/db93bdd05f964880a0d7f84977bc199d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:30:52.266655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c857e-d659-4b5c-81b4-a4726a4a22a5 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.266941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c857e-d659-4b5c-81b4-a4726a4a22a5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.267494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c857e-d659-4b5c-81b4-a4726a4a22a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.291040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-833c857e-d659-4b5c-81b4-a4726a4a22a5 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:52.292260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-833c857e-d659-4b5c-81b4-a4726a4a22a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:52.381169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1077/2134] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:52.390228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483a36-b2dc-446e-811c-bbe7c42f3516 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.390635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483a36-b2dc-446e-811c-bbe7c42f3516 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.391021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483a36-b2dc-446e-811c-bbe7c42f3516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.427177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-d6483a36-b2dc-446e-811c-bbe7c42f3516 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:52.428421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d6483a36-b2dc-446e-811c-bbe7c42f3516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:52.457561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1058/2135] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:52.457928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c857e-d659-4b5c-81b4-a4726a4a22a5 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:52.469569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c857e-d659-4b5c-81b4-a4726a4a22a5 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:52.476283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f62a-33c0-4381-80cb-d8bda44b0ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:52.476919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f62a-33c0-4381-80cb-d8bda44b0ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.477130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f62a-33c0-4381-80cb-d8bda44b0ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.477400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f62a-33c0-4381-80cb-d8bda44b0ee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.478092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f62a-33c0-4381-80cb-d8bda44b0ee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:52.478389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f62a-33c0-4381-80cb-d8bda44b0ee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:52.480208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f62a-33c0-4381-80cb-d8bda44b0ee4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:52.510717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1078/2136] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:52.516785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483a36-b2dc-446e-811c-bbe7c42f3516 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:52.533072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483a36-b2dc-446e-811c-bbe7c42f3516 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:52.539407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b3bb9-259f-467c-be9f-0c89f1e14fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:52.540140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b3bb9-259f-467c-be9f-0c89f1e14fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.540543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b3bb9-259f-467c-be9f-0c89f1e14fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.540926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b3bb9-259f-467c-be9f-0c89f1e14fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.541789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b3bb9-259f-467c-be9f-0c89f1e14fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:52.545285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b3bb9-259f-467c-be9f-0c89f1e14fe3 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:52.578442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4303f-2f84-4b79-9837-2137ea6b952f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.578752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4303f-2f84-4b79-9837-2137ea6b952f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.579315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4303f-2f84-4b79-9837-2137ea6b952f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.614563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1059/2137] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:52 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:52.618890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b3bb9-259f-467c-be9f-0c89f1e14fe3 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:52.647012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-48f4303f-2f84-4b79-9837-2137ea6b952f None None] MFA Rules not processed for user `b86acd50276842a3bc18e3ffa00e515a`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:52.648425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-48f4303f-2f84-4b79-9837-2137ea6b952f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:52.759989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b3bb9-259f-467c-be9f-0c89f1e14fe3 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:52.770801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37a964-cab6-41a9-9146-609c34ef1449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:52.771606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37a964-cab6-41a9-9146-609c34ef1449 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.771886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37a964-cab6-41a9-9146-609c34ef1449 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.772157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee37a964-cab6-41a9-9146-609c34ef1449 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.772828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37a964-cab6-41a9-9146-609c34ef1449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:52.774809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee37a964-cab6-41a9-9146-609c34ef1449 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:52.802311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1079/2138] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:52.820471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4303f-2f84-4b79-9837-2137ea6b952f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:52.822261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-48f4303f-2f84-4b79-9837-2137ea6b952f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:52.831316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1060/2139] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:52 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:52.835904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37a964-cab6-41a9-9146-609c34ef1449 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:52.849758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee37a964-cab6-41a9-9146-609c34ef1449 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:52.855976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a120f-cbc4-4daf-8726-c894e86b613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:52.856536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a120f-cbc4-4daf-8726-c894e86b613e admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.856983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a120f-cbc4-4daf-8726-c894e86b613e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.857855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a120f-cbc4-4daf-8726-c894e86b613e admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.858518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a120f-cbc4-4daf-8726-c894e86b613e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:52.860161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a120f-cbc4-4daf-8726-c894e86b613e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:52.871865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1061/2140] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:30:52.876797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a120f-cbc4-4daf-8726-c894e86b613e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:52.888339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a120f-cbc4-4daf-8726-c894e86b613e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:52.903170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93a4e0-b5ad-4457-b379-80bd8574baf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:52.904191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93a4e0-b5ad-4457-b379-80bd8574baf1 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.905266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93a4e0-b5ad-4457-b379-80bd8574baf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.905713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93a4e0-b5ad-4457-b379-80bd8574baf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.906557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93a4e0-b5ad-4457-b379-80bd8574baf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:52.908470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93a4e0-b5ad-4457-b379-80bd8574baf1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:52.910123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:52.926117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4303f-2f84-4b79-9837-2137ea6b952f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:52.931536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf652288-25c4-4a46-89ed-aa86ba6c5cbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:52.932185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf652288-25c4-4a46-89ed-aa86ba6c5cbb service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.932453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf652288-25c4-4a46-89ed-aa86ba6c5cbb service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.932763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf652288-25c4-4a46-89ed-aa86ba6c5cbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.937074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1062/2141] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:30:52.940962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93a4e0-b5ad-4457-b379-80bd8574baf1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:52.944717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf652288-25c4-4a46-89ed-aa86ba6c5cbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:52.946171 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf652288-25c4-4a46-89ed-aa86ba6c5cbb service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:52.953114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93a4e0-b5ad-4457-b379-80bd8574baf1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:52.957625 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1080/2142] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:52.958997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9a90e-aa01-4e5a-bc3a-8e23a8fb72fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:52.959969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9a90e-aa01-4e5a-bc3a-8e23a8fb72fd admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:52.960244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9a90e-aa01-4e5a-bc3a-8e23a8fb72fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:52.960538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9a90e-aa01-4e5a-bc3a-8e23a8fb72fd admin admin] PATH_INFO: `/v3/projects/a262dc3512aa4a2c9eaeca41dd9dbc6e/users/3a1c4a96b5cb4f1d94d8fdd414a6d35f/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:52.977520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9a90e-aa01-4e5a-bc3a-8e23a8fb72fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a262dc3512aa4a2c9eaeca41dd9dbc6e, user_id=3a1c4a96b5cb4f1d94d8fdd414a6d35f, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:52.980398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9a90e-aa01-4e5a-bc3a-8e23a8fb72fd admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:53.003837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1063/2143] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:52 2022] PUT /identity/v3/projects/a262dc3512aa4a2c9eaeca41dd9dbc6e/users/3a1c4a96b5cb4f1d94d8fdd414a6d35f/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:53.016869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f136f7-8ecf-438e-ae74-f8ffb5c4cda8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:53.017158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f136f7-8ecf-438e-ae74-f8ffb5c4cda8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:53.017612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f136f7-8ecf-438e-ae74-f8ffb5c4cda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:53.051298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a6f136f7-8ecf-438e-ae74-f8ffb5c4cda8 None None] MFA Rules not processed for user `3a1c4a96b5cb4f1d94d8fdd414a6d35f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:53.052579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a6f136f7-8ecf-438e-ae74-f8ffb5c4cda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:53.058044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:53.145292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1081/2144] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:53.165264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9a90e-aa01-4e5a-bc3a-8e23a8fb72fd admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:53.167671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-44f9a90e-aa01-4e5a-bc3a-8e23a8fb72fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:53.261436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9a90e-aa01-4e5a-bc3a-8e23a8fb72fd admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:53.268468 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773af83-5817-4938-a1a6-f1838a839921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:53.269219 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773af83-5817-4938-a1a6-f1838a839921 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:53.269413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773af83-5817-4938-a1a6-f1838a839921 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:53.269672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773af83-5817-4938-a1a6-f1838a839921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:53.281577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773af83-5817-4938-a1a6-f1838a839921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:53.282955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773af83-5817-4938-a1a6-f1838a839921 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:53.298952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1064/2145] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:54.435123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fb522-bef7-437b-8424-bbcfb5661d38 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:54.436328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fb522-bef7-437b-8424-bbcfb5661d38 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:54.436328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fb522-bef7-437b-8424-bbcfb5661d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:54.455897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0c6fb522-bef7-437b-8424-bbcfb5661d38 None None] MFA Rules not processed for user `b86acd50276842a3bc18e3ffa00e515a`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:54.457423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0c6fb522-bef7-437b-8424-bbcfb5661d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:54.529390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1082/2146] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:56.648190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773af83-5817-4938-a1a6-f1838a839921 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:56.720694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773af83-5817-4938-a1a6-f1838a839921 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:56.725520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:56.726146 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:56.726423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:56.726694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] PATH_INFO: `/v3/users/3a1c4a96b5cb4f1d94d8fdd414a6d35f` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:56.727809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a1c4a96b5cb4f1d94d8fdd414a6d35f)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:56.729759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:56.782043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a1c4a96b5cb4f1d94d8fdd414a6d35f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:56.782752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a1c4a96b5cb4f1d94d8fdd414a6d35f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:56.795241 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a1c4a96b5cb4f1d94d8fdd414a6d35f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:56.802519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a1c4a96b5cb4f1d94d8fdd414a6d35f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:56.805519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:56.812708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a1c4a96b5cb4f1d94d8fdd414a6d35f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:56.821559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1065/2147] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:30:56 2022] DELETE /identity/v3/users/3a1c4a96b5cb4f1d94d8fdd414a6d35f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:56.827345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027f42b-104f-4fdb-b1af-788fbb7472a3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:56.827823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027f42b-104f-4fdb-b1af-788fbb7472a3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:56.827917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027f42b-104f-4fdb-b1af-788fbb7472a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:56.854975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-8027f42b-104f-4fdb-b1af-788fbb7472a3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:56.856201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8027f42b-104f-4fdb-b1af-788fbb7472a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:56.917170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1083/2148] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:56.927848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:56.929511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:56.992905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa1514-eb5e-46ef-a2fe-d9cfaf3130aa admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:56.996838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6be0b-ba44-44e9-b5ae-1f9e36feed9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:56.997382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6be0b-ba44-44e9-b5ae-1f9e36feed9f service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:56.998384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6be0b-ba44-44e9-b5ae-1f9e36feed9f service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:56.998759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6be0b-ba44-44e9-b5ae-1f9e36feed9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:57.008557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a6be0b-ba44-44e9-b5ae-1f9e36feed9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:57.010012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a6be0b-ba44-44e9-b5ae-1f9e36feed9f service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:57.021233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1066/2149] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:30:57.487297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027f42b-104f-4fdb-b1af-788fbb7472a3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:57.489588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8027f42b-104f-4fdb-b1af-788fbb7472a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:57.502818 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027f42b-104f-4fdb-b1af-788fbb7472a3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:57.508945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:57.509710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:57.510095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:57.510473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] PATH_INFO: `/v3/projects/a262dc3512aa4a2c9eaeca41dd9dbc6e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:57.512085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a262dc3512aa4a2c9eaeca41dd9dbc6e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:57.515170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:57.576753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:57.591109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a262dc3512aa4a2c9eaeca41dd9dbc6e', 'request_id': 'req-5c086258-2bc8-44e7-8376-c6dbf2d10129'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:57.605713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a262dc3512aa4a2c9eaeca41dd9dbc6e', 'request_id': 'req-5c086258-2bc8-44e7-8376-c6dbf2d10129'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:57.612583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a262dc3512aa4a2c9eaeca41dd9dbc6e', 'request_id': 'req-5c086258-2bc8-44e7-8376-c6dbf2d10129'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:57.627428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] The token cache is being invalidate because project a262dc3512aa4a2c9eaeca41dd9dbc6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:30:57.627722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:57.628374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5c086258-2bc8-44e7-8376-c6dbf2d10129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:30:57.630095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1084/2150] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:30:57 2022] DELETE /identity/v3/projects/a262dc3512aa4a2c9eaeca41dd9dbc6e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:57.637179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e7bc-7ec5-44d6-8fa8-3246dfb0f5ee None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:57.637465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e7bc-7ec5-44d6-8fa8-3246dfb0f5ee None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:57.637732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15e7bc-7ec5-44d6-8fa8-3246dfb0f5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:57.667684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ad15e7bc-7ec5-44d6-8fa8-3246dfb0f5ee None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:57.668943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ad15e7bc-7ec5-44d6-8fa8-3246dfb0f5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:57.782524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1067/2151] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:57.792320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5e200-1671-49e5-b9f3-15de6fb503f5 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:57.792707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5e200-1671-49e5-b9f3-15de6fb503f5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:57.793188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5e200-1671-49e5-b9f3-15de6fb503f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:57.831893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-72a5e200-1671-49e5-b9f3-15de6fb503f5 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:57.833632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-72a5e200-1671-49e5-b9f3-15de6fb503f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:57.845799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1085/2152] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:30:57.852012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e7bc-7ec5-44d6-8fa8-3246dfb0f5ee None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:57.865470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15e7bc-7ec5-44d6-8fa8-3246dfb0f5ee None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:57.873626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e63d0-4ee7-4159-abab-acf5f12410cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:57.874510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e63d0-4ee7-4159-abab-acf5f12410cf admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:57.875007 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e63d0-4ee7-4159-abab-acf5f12410cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:57.875467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e63d0-4ee7-4159-abab-acf5f12410cf admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:57.876318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e63d0-4ee7-4159-abab-acf5f12410cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:30:57.876824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e63d0-4ee7-4159-abab-acf5f12410cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:57.878917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e63d0-4ee7-4159-abab-acf5f12410cf admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:57.903707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1068/2153] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:30:57.913226 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5e200-1671-49e5-b9f3-15de6fb503f5 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:57.936194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5e200-1671-49e5-b9f3-15de6fb503f5 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:57.946984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1a63f-9413-4691-8224-060855b25e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:57.947882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1a63f-9413-4691-8224-060855b25e72 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:57.948364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1a63f-9413-4691-8224-060855b25e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:57.948886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1a63f-9413-4691-8224-060855b25e72 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:57.949720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1a63f-9413-4691-8224-060855b25e72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:57.953139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1a63f-9413-4691-8224-060855b25e72 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:58.007307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e63d0-4ee7-4159-abab-acf5f12410cf admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:58.008624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1086/2154] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:30:57 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:30:58.018766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e63d0-4ee7-4159-abab-acf5f12410cf admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:58.087392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb0763-60be-4421-85e2-9df438e79e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:58.088082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb0763-60be-4421-85e2-9df438e79e72 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:58.088448 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb0763-60be-4421-85e2-9df438e79e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:58.088817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb0763-60be-4421-85e2-9df438e79e72 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:58.089518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb0763-60be-4421-85e2-9df438e79e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:58.091979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb0763-60be-4421-85e2-9df438e79e72 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:58.115417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1069/2155] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:30:58 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:30:58.119535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1a63f-9413-4691-8224-060855b25e72 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:58.133079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1a63f-9413-4691-8224-060855b25e72 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:58.138427 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0bd3b-bc9d-4ea6-a798-b964e27271b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:58.139050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0bd3b-bc9d-4ea6-a798-b964e27271b8 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:58.139332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0bd3b-bc9d-4ea6-a798-b964e27271b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:58.139598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0bd3b-bc9d-4ea6-a798-b964e27271b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:58.140241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0bd3b-bc9d-4ea6-a798-b964e27271b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:58.142116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0bd3b-bc9d-4ea6-a798-b964e27271b8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:58.153892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1087/2156] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:30:58.158002 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb0763-60be-4421-85e2-9df438e79e72 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:58.168563 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb0763-60be-4421-85e2-9df438e79e72 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:58.174179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f469fca-dcbf-4aad-8b8f-c47a07d84d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:58.174807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f469fca-dcbf-4aad-8b8f-c47a07d84d2c admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:58.175086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f469fca-dcbf-4aad-8b8f-c47a07d84d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:58.175398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f469fca-dcbf-4aad-8b8f-c47a07d84d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:58.176005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f469fca-dcbf-4aad-8b8f-c47a07d84d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:58.177660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f469fca-dcbf-4aad-8b8f-c47a07d84d2c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:58.192524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1070/2157] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:30:58.196660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0bd3b-bc9d-4ea6-a798-b964e27271b8 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:58.238823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0bd3b-bc9d-4ea6-a798-b964e27271b8 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:58.255911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d8490-186c-4b9e-bc44-3117ea281080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:58.256610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d8490-186c-4b9e-bc44-3117ea281080 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:58.256894 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d8490-186c-4b9e-bc44-3117ea281080 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:58.257167 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d8490-186c-4b9e-bc44-3117ea281080 admin admin] PATH_INFO: `/v3/projects/3f39cdb9221440658d7cd52c341cf08a/users/af0e23f782c74c5d896ab24b64b43b18/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:58.275228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d8490-186c-4b9e-bc44-3117ea281080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f39cdb9221440658d7cd52c341cf08a, user_id=af0e23f782c74c5d896ab24b64b43b18, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:58.277967 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d8490-186c-4b9e-bc44-3117ea281080 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:58.303964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1088/2158] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:30:58 2022] PUT /identity/v3/projects/3f39cdb9221440658d7cd52c341cf08a/users/af0e23f782c74c5d896ab24b64b43b18/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:30:58.309179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605d869-8a3d-449e-a396-aa37f31c3fcf None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:58.309464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605d869-8a3d-449e-a396-aa37f31c3fcf None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:58.309769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605d869-8a3d-449e-a396-aa37f31c3fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:58.317243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:30:58.333679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-9605d869-8a3d-449e-a396-aa37f31c3fcf None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:30:58.334767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9605d869-8a3d-449e-a396-aa37f31c3fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:58.400352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1071/2159] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:30:58.411159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d8490-186c-4b9e-bc44-3117ea281080 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:30:58.412891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3d6d8490-186c-4b9e-bc44-3117ea281080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:30:58.471279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d8490-186c-4b9e-bc44-3117ea281080 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:30:58.475584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d743a2-1e87-4150-9dc9-1a6eef04c883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:30:58.476197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d743a2-1e87-4150-9dc9-1a6eef04c883 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:30:58.476468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d743a2-1e87-4150-9dc9-1a6eef04c883 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:30:58.476771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d743a2-1e87-4150-9dc9-1a6eef04c883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:30:58.487222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d743a2-1e87-4150-9dc9-1a6eef04c883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:30:58.488780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d743a2-1e87-4150-9dc9-1a6eef04c883 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:30:58.498562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1089/2160] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:30:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:31:04.254930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605d869-8a3d-449e-a396-aa37f31c3fcf None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:04.256572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9605d869-8a3d-449e-a396-aa37f31c3fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:04.274242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605d869-8a3d-449e-a396-aa37f31c3fcf None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:04.280414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:04.281099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:04.281911 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:04.282197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] PATH_INFO: `/v3/users/6f67deff06894bffb9ac99680df14b88` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:04.283394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f67deff06894bffb9ac99680df14b88)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:04.285439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:04.389755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f67deff06894bffb9ac99680df14b88'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:04.390419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f67deff06894bffb9ac99680df14b88'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:04.410420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f67deff06894bffb9ac99680df14b88'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:04.417554 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f67deff06894bffb9ac99680df14b88'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:04.426454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-91a811cb-6fe5-410b-81bc-d0fe00d9b5fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f67deff06894bffb9ac99680df14b88'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:04.441123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1072/2161] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:31:04 2022] DELETE /identity/v3/users/6f67deff06894bffb9ac99680df14b88 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:31:04.864615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d743a2-1e87-4150-9dc9-1a6eef04c883 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:04.891459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d743a2-1e87-4150-9dc9-1a6eef04c883 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:04.990981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:04.991901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:04.992249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:04.992620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] PATH_INFO: `/v3/projects/eb6a951afecc44219772f82bb09f91e3` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:04.993859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb6a951afecc44219772f82bb09f91e3)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:04.996438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.019161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:05.080756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb6a951afecc44219772f82bb09f91e3', 'request_id': 'req-4789ac46-967e-4346-af4a-9ecdd071eeda'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:05.088390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb6a951afecc44219772f82bb09f91e3', 'request_id': 'req-4789ac46-967e-4346-af4a-9ecdd071eeda'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:05.095755 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb6a951afecc44219772f82bb09f91e3', 'request_id': 'req-4789ac46-967e-4346-af4a-9ecdd071eeda'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:05.115675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] The token cache is being invalidate because project eb6a951afecc44219772f82bb09f91e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:31:05.116046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:05.116899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4789ac46-967e-4346-af4a-9ecdd071eeda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:05.119018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1090/2162] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:31:04 2022] DELETE /identity/v3/projects/eb6a951afecc44219772f82bb09f91e3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:31:05.125291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058acab-e960-40d4-8c77-c01137a7bf06 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.125972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058acab-e960-40d4-8c77-c01137a7bf06 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.126253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058acab-e960-40d4-8c77-c01137a7bf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.148172 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3058acab-e960-40d4-8c77-c01137a7bf06 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:05.149850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3058acab-e960-40d4-8c77-c01137a7bf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:05.173798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0eb5-3adb-4ca9-bc67-96ee8d4acfe2 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.174216 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0eb5-3adb-4ca9-bc67-96ee8d4acfe2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.174597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0eb5-3adb-4ca9-bc67-96ee8d4acfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.226841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-26be0eb5-3adb-4ca9-bc67-96ee8d4acfe2 None None] MFA Rules not processed for user `af0e23f782c74c5d896ab24b64b43b18`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:05.242121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-26be0eb5-3adb-4ca9-bc67-96ee8d4acfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:05.260284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1073/2163] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:31:05.268597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632eaab-9654-40b5-a940-5cdc1fe58039 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.268977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632eaab-9654-40b5-a940-5cdc1fe58039 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.269320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632eaab-9654-40b5-a940-5cdc1fe58039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.293485 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1632eaab-9654-40b5-a940-5cdc1fe58039 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:05.295495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1632eaab-9654-40b5-a940-5cdc1fe58039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:05.307813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1074/2164] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:31:05.312025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632eaab-9654-40b5-a940-5cdc1fe58039 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.324009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632eaab-9654-40b5-a940-5cdc1fe58039 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.330793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a8f0-5f4f-4b96-a99b-537472ca9ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.332246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a8f0-5f4f-4b96-a99b-537472ca9ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.332746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a8f0-5f4f-4b96-a99b-537472ca9ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.333080 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1091/2165] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:31:05.333525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a8f0-5f4f-4b96-a99b-537472ca9ca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.334493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64a8f0-5f4f-4b96-a99b-537472ca9ca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:31:05.334949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64a8f0-5f4f-4b96-a99b-537472ca9ca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.336998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64a8f0-5f4f-4b96-a99b-537472ca9ca4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.337419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be0eb5-3adb-4ca9-bc67-96ee8d4acfe2 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.338866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-26be0eb5-3adb-4ca9-bc67-96ee8d4acfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:05.340160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:05.351493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1075/2166] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:31:05.355806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a8f0-5f4f-4b96-a99b-537472ca9ca4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.358167 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be0eb5-3adb-4ca9-bc67-96ee8d4acfe2 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.363172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63361e34-5107-46b4-85fb-8d9ed4856116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.363810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63361e34-5107-46b4-85fb-8d9ed4856116 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.364112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63361e34-5107-46b4-85fb-8d9ed4856116 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.364382 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63361e34-5107-46b4-85fb-8d9ed4856116 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.365058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63361e34-5107-46b4-85fb-8d9ed4856116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.366987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63361e34-5107-46b4-85fb-8d9ed4856116 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.370128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a8f0-5f4f-4b96-a99b-537472ca9ca4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.376318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b16b07-a9c0-4e9a-8d3d-a02111535bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.376971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b16b07-a9c0-4e9a-8d3d-a02111535bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.377252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b16b07-a9c0-4e9a-8d3d-a02111535bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.377525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b16b07-a9c0-4e9a-8d3d-a02111535bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.378164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b16b07-a9c0-4e9a-8d3d-a02111535bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.380100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b16b07-a9c0-4e9a-8d3d-a02111535bc9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.402239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1092/2167] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:31:05 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:31:05.407154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63361e34-5107-46b4-85fb-8d9ed4856116 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.422138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1076/2168] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:31:05 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:31:05.423077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b16b07-a9c0-4e9a-8d3d-a02111535bc9 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.516869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b16b07-a9c0-4e9a-8d3d-a02111535bc9 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.521985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67f89-e994-4c43-bda3-2ac64a67c37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.522724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67f89-e994-4c43-bda3-2ac64a67c37c admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.523008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67f89-e994-4c43-bda3-2ac64a67c37c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.523281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d67f89-e994-4c43-bda3-2ac64a67c37c admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.523927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d67f89-e994-4c43-bda3-2ac64a67c37c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.524794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63361e34-5107-46b4-85fb-8d9ed4856116 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.525948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d67f89-e994-4c43-bda3-2ac64a67c37c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.536178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9a986-e4a3-44a5-8fe9-3eea5f8a808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.537094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9a986-e4a3-44a5-8fe9-3eea5f8a808f admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.537482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9a986-e4a3-44a5-8fe9-3eea5f8a808f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.537689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9a986-e4a3-44a5-8fe9-3eea5f8a808f admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.538365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9a986-e4a3-44a5-8fe9-3eea5f8a808f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.541415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a9a986-e4a3-44a5-8fe9-3eea5f8a808f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.560943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1077/2169] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:31:05 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:31:05.562102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67f89-e994-4c43-bda3-2ac64a67c37c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.573152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:05.575029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d67f89-e994-4c43-bda3-2ac64a67c37c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.585722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48602a5a-c103-489e-a4d0-db4001a63740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.586332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48602a5a-c103-489e-a4d0-db4001a63740 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.586747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48602a5a-c103-489e-a4d0-db4001a63740 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.587045 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48602a5a-c103-489e-a4d0-db4001a63740 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.587709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48602a5a-c103-489e-a4d0-db4001a63740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.589395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48602a5a-c103-489e-a4d0-db4001a63740 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.594739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1093/2170] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:31:05 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:31:05.595669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9a986-e4a3-44a5-8fe9-3eea5f8a808f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.614015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1078/2171] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:31:05.616170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48602a5a-c103-489e-a4d0-db4001a63740 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.617784 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a9a986-e4a3-44a5-8fe9-3eea5f8a808f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.635462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48602a5a-c103-489e-a4d0-db4001a63740 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.637396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60def-fd0e-4ff7-a848-163f88e08aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.638246 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60def-fd0e-4ff7-a848-163f88e08aba admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.638450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60def-fd0e-4ff7-a848-163f88e08aba admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.638850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece60def-fd0e-4ff7-a848-163f88e08aba admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.639563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece60def-fd0e-4ff7-a848-163f88e08aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.641399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece60def-fd0e-4ff7-a848-163f88e08aba admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.641943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77867721-8cc1-4c6b-8420-31c5ffa89bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.642669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867721-8cc1-4c6b-8420-31c5ffa89bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.643050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867721-8cc1-4c6b-8420-31c5ffa89bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.643423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867721-8cc1-4c6b-8420-31c5ffa89bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.644174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77867721-8cc1-4c6b-8420-31c5ffa89bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.646208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77867721-8cc1-4c6b-8420-31c5ffa89bb0 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.653076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1094/2172] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:31:05.657306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60def-fd0e-4ff7-a848-163f88e08aba admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.667306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1079/2173] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:31:05.668127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece60def-fd0e-4ff7-a848-163f88e08aba admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.673364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77867721-8cc1-4c6b-8420-31c5ffa89bb0 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.675549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d0ac2-ea11-46af-8d8f-9df20a076348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.676221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d0ac2-ea11-46af-8d8f-9df20a076348 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.676520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d0ac2-ea11-46af-8d8f-9df20a076348 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.676814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d0ac2-ea11-46af-8d8f-9df20a076348 admin admin] PATH_INFO: `/v3/projects/a4f6de578de64bcb8175fe1bd87b025e/users/6e123c6435e746eab4d3bf8d425f3186/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.685526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77867721-8cc1-4c6b-8420-31c5ffa89bb0 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.691091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257809e-c41c-475a-b6da-2d64e19b84da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.691967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257809e-c41c-475a-b6da-2d64e19b84da admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.692306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257809e-c41c-475a-b6da-2d64e19b84da admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.692757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257809e-c41c-475a-b6da-2d64e19b84da admin admin] PATH_INFO: `/v3/projects/b1bcb33a579e47518fafa960505447dc/users/e7a36e4630904f4eac08c9de1b8235ca/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.694061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d0ac2-ea11-46af-8d8f-9df20a076348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f6de578de64bcb8175fe1bd87b025e, user_id=6e123c6435e746eab4d3bf8d425f3186, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.696875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d0ac2-ea11-46af-8d8f-9df20a076348 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.712459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1095/2174] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:31:05 2022] PUT /identity/v3/projects/a4f6de578de64bcb8175fe1bd87b025e/users/6e123c6435e746eab4d3bf8d425f3186/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:31:05.715116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257809e-c41c-475a-b6da-2d64e19b84da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bcb33a579e47518fafa960505447dc, user_id=e7a36e4630904f4eac08c9de1b8235ca, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.716837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d0ac2-ea11-46af-8d8f-9df20a076348 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.719596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257809e-c41c-475a-b6da-2d64e19b84da admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.740553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1080/2175] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:31:05 2022] PUT /identity/v3/projects/b1bcb33a579e47518fafa960505447dc/users/e7a36e4630904f4eac08c9de1b8235ca/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:31:05.743291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4468c-9c4d-41fa-8377-c5ae6d643b1f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.743633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4468c-9c4d-41fa-8377-c5ae6d643b1f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.743863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4468c-9c4d-41fa-8377-c5ae6d643b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.764980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fce4468c-9c4d-41fa-8377-c5ae6d643b1f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:05.766223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fce4468c-9c4d-41fa-8377-c5ae6d643b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:05.801021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d0ac2-ea11-46af-8d8f-9df20a076348 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.838844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1081/2176] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:31:05.854172 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4468c-9c4d-41fa-8377-c5ae6d643b1f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.856063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fce4468c-9c4d-41fa-8377-c5ae6d643b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:05.881401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:05.887317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c073-218c-478b-b731-8327d2f13a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.888338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c073-218c-478b-b731-8327d2f13a04 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.888847 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c073-218c-478b-b731-8327d2f13a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.889191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66c073-218c-478b-b731-8327d2f13a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.889911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66c073-218c-478b-b731-8327d2f13a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.891632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66c073-218c-478b-b731-8327d2f13a04 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.909241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1096/2177] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:31:05.915840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c073-218c-478b-b731-8327d2f13a04 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:05.928857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66c073-218c-478b-b731-8327d2f13a04 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.933582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4468c-9c4d-41fa-8377-c5ae6d643b1f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:05.935906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533fd05-44b3-4621-ba24-4baff9dc027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.936778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533fd05-44b3-4621-ba24-4baff9dc027a admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.937131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533fd05-44b3-4621-ba24-4baff9dc027a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.937515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533fd05-44b3-4621-ba24-4baff9dc027a admin admin] PATH_INFO: `/v3/projects/a4f6de578de64bcb8175fe1bd87b025e/users/6e123c6435e746eab4d3bf8d425f3186/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.938972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a460a77-9625-48f9-9daa-ae82c6640e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:05.939687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a460a77-9625-48f9-9daa-ae82c6640e68 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:05.939813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533fd05-44b3-4621-ba24-4baff9dc027a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f6de578de64bcb8175fe1bd87b025e, user_id=6e123c6435e746eab4d3bf8d425f3186, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.939978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a460a77-9625-48f9-9daa-ae82c6640e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:05.940424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a460a77-9625-48f9-9daa-ae82c6640e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:05.942790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533fd05-44b3-4621-ba24-4baff9dc027a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.955407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a460a77-9625-48f9-9daa-ae82c6640e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:05.955795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1097/2178] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:31:05 2022] PUT /identity/v3/projects/a4f6de578de64bcb8175fe1bd87b025e/users/6e123c6435e746eab4d3bf8d425f3186/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:31:05.957679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a460a77-9625-48f9-9daa-ae82c6640e68 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:05.969932 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1082/2179] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:31:14.886943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533fd05-44b3-4621-ba24-4baff9dc027a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:14.895285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0533fd05-44b3-4621-ba24-4baff9dc027a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:15.071921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:15.087564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533fd05-44b3-4621-ba24-4baff9dc027a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:15.104295 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:15.105006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:15.105287 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:15.105563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] PATH_INFO: `/v3/users/27553f7a13e54b0c8758ca64d6229b78` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:15.111388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=27553f7a13e54b0c8758ca64d6229b78)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:15.113373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:15.217263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27553f7a13e54b0c8758ca64d6229b78'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:15.218399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27553f7a13e54b0c8758ca64d6229b78'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:15.245294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27553f7a13e54b0c8758ca64d6229b78'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:15.270087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27553f7a13e54b0c8758ca64d6229b78'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:15.275695 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27553f7a13e54b0c8758ca64d6229b78'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:15.337499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1098/2180] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:31:14 2022] DELETE /identity/v3/users/27553f7a13e54b0c8758ca64d6229b78 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:31:15.342124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4df3c9-1599-4cb9-bc42-f8f65d050086 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:15.342626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4df3c9-1599-4cb9-bc42-f8f65d050086 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:15.342797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4df3c9-1599-4cb9-bc42-f8f65d050086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:15.406731 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ec4df3c9-1599-4cb9-bc42-f8f65d050086 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:15.407991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ec4df3c9-1599-4cb9-bc42-f8f65d050086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:15.598038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1083/2181] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:31:15.614173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:15.616467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:15.755634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce956dd-1f93-45cb-a6ab-ab70aa9ff771 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:15.755920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce956dd-1f93-45cb-a6ab-ab70aa9ff771 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:15.756265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce956dd-1f93-45cb-a6ab-ab70aa9ff771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:15.783350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf9747-497a-498a-b6fc-2bdb1ce94d8a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:15.787874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379c085-2bfe-4cb8-936e-c0fd38aa6ec7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:15.788610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379c085-2bfe-4cb8-936e-c0fd38aa6ec7 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:15.789322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379c085-2bfe-4cb8-936e-c0fd38aa6ec7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:15.789435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-8ce956dd-1f93-45cb-a6ab-ab70aa9ff771 None None] MFA Rules not processed for user `6e123c6435e746eab4d3bf8d425f3186`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:15.789619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379c085-2bfe-4cb8-936e-c0fd38aa6ec7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:15.790358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8ce956dd-1f93-45cb-a6ab-ab70aa9ff771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:15.813014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379c085-2bfe-4cb8-936e-c0fd38aa6ec7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:15.816051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379c085-2bfe-4cb8-936e-c0fd38aa6ec7 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:15.857380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1099/2182] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:31:15.861366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eae43-0b76-4a67-8032-8e5e32cb7c1a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:15.866592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eae43-0b76-4a67-8032-8e5e32cb7c1a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:15.866978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70eae43-0b76-4a67-8032-8e5e32cb7c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:15.909226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1084/2183] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:31:15.924547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f70eae43-0b76-4a67-8032-8e5e32cb7c1a None None] MFA Rules not processed for user `e7a36e4630904f4eac08c9de1b8235ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:15.925872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f70eae43-0b76-4a67-8032-8e5e32cb7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:15.926417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce956dd-1f93-45cb-a6ab-ab70aa9ff771 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:15.931419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8ce956dd-1f93-45cb-a6ab-ab70aa9ff771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:16.042518 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1100/2184] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:31:16.074840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eae43-0b76-4a67-8032-8e5e32cb7c1a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:16.076454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f70eae43-0b76-4a67-8032-8e5e32cb7c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:16.077861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:16.093364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:16.139385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce956dd-1f93-45cb-a6ab-ab70aa9ff771 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:16.153930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d6f5e-0c5f-434e-852c-f9ae04c027a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:16.154576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d6f5e-0c5f-434e-852c-f9ae04c027a3 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:16.154858 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d6f5e-0c5f-434e-852c-f9ae04c027a3 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:16.155130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d6f5e-0c5f-434e-852c-f9ae04c027a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:16.161975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3c4d6f5e-0c5f-434e-852c-f9ae04c027a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:16.237309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70eae43-0b76-4a67-8032-8e5e32cb7c1a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:16.244153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd71cca-38b6-4e27-8a36-964dc635a82c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:16.244962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd71cca-38b6-4e27-8a36-964dc635a82c service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:16.245542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd71cca-38b6-4e27-8a36-964dc635a82c service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:16.245900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd71cca-38b6-4e27-8a36-964dc635a82c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:16.253953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d6f5e-0c5f-434e-852c-f9ae04c027a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:16.256785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd71cca-38b6-4e27-8a36-964dc635a82c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:16.258492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd71cca-38b6-4e27-8a36-964dc635a82c service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:16.259085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d6f5e-0c5f-434e-852c-f9ae04c027a3 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:16.271141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1101/2185] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:31:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:31:16.272970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1085/2186] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:31:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:31:16.376562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd71cca-38b6-4e27-8a36-964dc635a82c service nova] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:16.378481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fcd71cca-38b6-4e27-8a36-964dc635a82c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:16.396389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd71cca-38b6-4e27-8a36-964dc635a82c service nova] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:16.413505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:16.414178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:16.414448 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:16.414727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] PATH_INFO: `/v3/projects/13ec47a00b094822929aa10ecff3693d` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:16.415918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ec47a00b094822929aa10ecff3693d)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:16.419417 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:16.520082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ec47a00b094822929aa10ecff3693d', 'request_id': 'req-385368e4-4883-4edb-ae01-c6848869b271'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:16.533595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ec47a00b094822929aa10ecff3693d', 'request_id': 'req-385368e4-4883-4edb-ae01-c6848869b271'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:16.548399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ec47a00b094822929aa10ecff3693d', 'request_id': 'req-385368e4-4883-4edb-ae01-c6848869b271'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:16.576683 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] The token cache is being invalidate because project 13ec47a00b094822929aa10ecff3693d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:31:16.577229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:16.578197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-385368e4-4883-4edb-ae01-c6848869b271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:16.598935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1102/2187] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:31:16 2022] DELETE /identity/v3/projects/13ec47a00b094822929aa10ecff3693d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:31:16.807936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dca3f-8d4c-4db3-ac9d-8f0c4628214b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:16.809124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dca3f-8d4c-4db3-ac9d-8f0c4628214b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:16.809666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dca3f-8d4c-4db3-ac9d-8f0c4628214b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:16.865742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-6c8dca3f-8d4c-4db3-ac9d-8f0c4628214b None None] MFA Rules not processed for user `b86acd50276842a3bc18e3ffa00e515a`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:16.869510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6c8dca3f-8d4c-4db3-ac9d-8f0c4628214b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:17.010368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1086/2188] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:31:18.275680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8b182-329f-41a6-aaae-940f6ec833a6 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:18.275680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8b182-329f-41a6-aaae-940f6ec833a6 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:18.276377 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8b182-329f-41a6-aaae-940f6ec833a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:18.307384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6aa8b182-329f-41a6-aaae-940f6ec833a6 None None] MFA Rules not processed for user `af0e23f782c74c5d896ab24b64b43b18`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:18.316831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6aa8b182-329f-41a6-aaae-940f6ec833a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:18.448629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1103/2189] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:31:18.831988 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395fc5b-6c29-4c4e-ab58-709c897d17ca None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:18.832902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395fc5b-6c29-4c4e-ab58-709c897d17ca None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:18.833332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395fc5b-6c29-4c4e-ab58-709c897d17ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:18.870972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-6395fc5b-6c29-4c4e-ab58-709c897d17ca None None] MFA Rules not processed for user `e7a36e4630904f4eac08c9de1b8235ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:18.873221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6395fc5b-6c29-4c4e-ab58-709c897d17ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:18.966229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1087/2190] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:31:28.430995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:28.954787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=85 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:35.221684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8b182-329f-41a6-aaae-940f6ec833a6 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:35.223205 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6aa8b182-329f-41a6-aaae-940f6ec833a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:35.294910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8b182-329f-41a6-aaae-940f6ec833a6 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:35.300775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:35.301405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:35.301679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:35.301950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] PATH_INFO: `/v3/users/894bb815eb0d424aa01ebc941a2349c6` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:35.303063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=894bb815eb0d424aa01ebc941a2349c6)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:35.305024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:35.357308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894bb815eb0d424aa01ebc941a2349c6'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:35.357958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894bb815eb0d424aa01ebc941a2349c6'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:35.365000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894bb815eb0d424aa01ebc941a2349c6'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:35.372402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894bb815eb0d424aa01ebc941a2349c6'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:35.373061 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894bb815eb0d424aa01ebc941a2349c6'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:35.390384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1104/2191] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:31:35 2022] DELETE /identity/v3/users/894bb815eb0d424aa01ebc941a2349c6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:31:35.394984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b1bd3-bf98-4303-9614-940bbacd644b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:35.395454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b1bd3-bf98-4303-9614-940bbacd644b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:35.395543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b1bd3-bf98-4303-9614-940bbacd644b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:35.415035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fd2b1bd3-bf98-4303-9614-940bbacd644b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:35.416307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fd2b1bd3-bf98-4303-9614-940bbacd644b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:35.475544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1088/2192] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:31:35.485519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:35.487044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:35.550640 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb45af-6997-4753-bfa6-8fb2148caa21 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:35.554315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12fb20-5d9f-445f-8da4-a45a3da05d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:35.554916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12fb20-5d9f-445f-8da4-a45a3da05d39 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:35.555193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12fb20-5d9f-445f-8da4-a45a3da05d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:35.555458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12fb20-5d9f-445f-8da4-a45a3da05d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:35.565286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12fb20-5d9f-445f-8da4-a45a3da05d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:35.566737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12fb20-5d9f-445f-8da4-a45a3da05d39 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:35.576783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1105/2193] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:31:35.920168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b1bd3-bf98-4303-9614-940bbacd644b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:35.921754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fd2b1bd3-bf98-4303-9614-940bbacd644b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:35.938220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b1bd3-bf98-4303-9614-940bbacd644b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:35.945004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:35.945651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:35.945930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:35.946202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] PATH_INFO: `/v3/projects/38d02d083ec24587add6fa8c2ae389a5` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:35.947351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d02d083ec24587add6fa8c2ae389a5)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:35.949404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:36.005635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d02d083ec24587add6fa8c2ae389a5', 'request_id': 'req-aba2da45-5f84-4035-a0b2-604b17ac17b4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:36.012073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d02d083ec24587add6fa8c2ae389a5', 'request_id': 'req-aba2da45-5f84-4035-a0b2-604b17ac17b4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:36.018304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d02d083ec24587add6fa8c2ae389a5', 'request_id': 'req-aba2da45-5f84-4035-a0b2-604b17ac17b4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:36.031187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] The token cache is being invalidate because project 38d02d083ec24587add6fa8c2ae389a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:31:36.031477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:36.032101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:31:36.033667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1089/2194] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:31:35 2022] DELETE /identity/v3/projects/38d02d083ec24587add6fa8c2ae389a5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:31:42.483893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fca4b1-bd07-4944-8b3c-88dff243dcc7 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:42.484307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fca4b1-bd07-4944-8b3c-88dff243dcc7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:42.484489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fca4b1-bd07-4944-8b3c-88dff243dcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:42.503968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d0fca4b1-bd07-4944-8b3c-88dff243dcc7 None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:42.505213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d0fca4b1-bd07-4944-8b3c-88dff243dcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:42.522790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:42.567152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1106/2195] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:31:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:31:42.611478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:31:42.612996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:42.691115 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2da45-5f84-4035-a0b2-604b17ac17b4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:31:42.697046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e62f09-7937-4803-863d-654e713d799d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:31:42.697868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e62f09-7937-4803-863d-654e713d799d service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:42.698181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e62f09-7937-4803-863d-654e713d799d service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:42.698455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e62f09-7937-4803-863d-654e713d799d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:42.709282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e62f09-7937-4803-863d-654e713d799d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:31:42.710748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e62f09-7937-4803-863d-654e713d799d service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:31:42.719802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1090/2196] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:31:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:31:52.560453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=15 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:52.716824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:31:59.855978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a93c4-20b5-495a-90b6-d9ebd1203f84 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:31:59.856547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a93c4-20b5-495a-90b6-d9ebd1203f84 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:31:59.856547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a93c4-20b5-495a-90b6-d9ebd1203f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:31:59.875705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b54a93c4-20b5-495a-90b6-d9ebd1203f84 None None] MFA Rules not processed for user `a6f08eaef2d149a9b0236d4bcbe03a92`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:31:59.876901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b54a93c4-20b5-495a-90b6-d9ebd1203f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:31:59.934761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1107/2197] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:31:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:00.585111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c795f-a2f0-4068-ab37-100bf7a62559 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:00.585635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c795f-a2f0-4068-ab37-100bf7a62559 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:00.585729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c795f-a2f0-4068-ab37-100bf7a62559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:00.607010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3f3c795f-a2f0-4068-ab37-100bf7a62559 None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:00.608242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3f3c795f-a2f0-4068-ab37-100bf7a62559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:00.616709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1091/2198] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 00:32:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:00.631388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a93c4-20b5-495a-90b6-d9ebd1203f84 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:00.647137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a93c4-20b5-495a-90b6-d9ebd1203f84 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:00.655151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd862a5-7922-4b96-bbc7-d921e89011f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:00.655842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd862a5-7922-4b96-bbc7-d921e89011f2 service swift] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:00.656214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd862a5-7922-4b96-bbc7-d921e89011f2 service swift] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:00.656917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd862a5-7922-4b96-bbc7-d921e89011f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:00.670454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd862a5-7922-4b96-bbc7-d921e89011f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:00.672734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd862a5-7922-4b96-bbc7-d921e89011f2 service swift] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:00.682253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1108/2199] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:32:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:32:03.055995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c795f-a2f0-4068-ab37-100bf7a62559 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:03.057640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3f3c795f-a2f0-4068-ab37-100bf7a62559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:03.130083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c795f-a2f0-4068-ab37-100bf7a62559 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:03.134703 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:03.135236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:03.135513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:03.135789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] PATH_INFO: `/v3/users/b86acd50276842a3bc18e3ffa00e515a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:03.137603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86acd50276842a3bc18e3ffa00e515a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:03.139474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:03.195551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86acd50276842a3bc18e3ffa00e515a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.196205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86acd50276842a3bc18e3ffa00e515a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.209111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86acd50276842a3bc18e3ffa00e515a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.216643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86acd50276842a3bc18e3ffa00e515a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.224154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b7e54516-ab48-468f-96c5-0a98f9c3f3d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86acd50276842a3bc18e3ffa00e515a'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.231481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1092/2200] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:32:03 2022] DELETE /identity/v3/users/b86acd50276842a3bc18e3ffa00e515a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:32:03.485805 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd862a5-7922-4b96-bbc7-d921e89011f2 service swift] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:03.499417 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd862a5-7922-4b96-bbc7-d921e89011f2 service swift] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:03.573678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:03.574433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:03.574777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:03.575777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] PATH_INFO: `/v3/projects/32e68213ef7041a79aaaf1d255d2395e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:03.577665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e68213ef7041a79aaaf1d255d2395e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:03.579605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:03.637751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e68213ef7041a79aaaf1d255d2395e', 'request_id': 'req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.645462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e68213ef7041a79aaaf1d255d2395e', 'request_id': 'req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.655512 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e68213ef7041a79aaaf1d255d2395e', 'request_id': 'req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.673617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] The token cache is being invalidate because project 32e68213ef7041a79aaaf1d255d2395e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:32:03.674500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.676284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:03.678164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1109/2201] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:32:03 2022] DELETE /identity/v3/projects/32e68213ef7041a79aaaf1d255d2395e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:32:03.682846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fe8cf-a9e3-495a-937b-f2f0017f6364 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:03.683188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fe8cf-a9e3-495a-937b-f2f0017f6364 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:03.683510 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fe8cf-a9e3-495a-937b-f2f0017f6364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:03.707556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-319fe8cf-a9e3-495a-937b-f2f0017f6364 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:03.709097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-319fe8cf-a9e3-495a-937b-f2f0017f6364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:03.727922 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:03.729798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:03.763224 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:32:03.789756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1093/2202] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:03.797629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b5eab-a98f-4793-b887-1e5e47dc43c9 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:03.797925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b5eab-a98f-4793-b887-1e5e47dc43c9 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:03.798201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b5eab-a98f-4793-b887-1e5e47dc43c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:03.810387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9080faa-9b0d-4e0c-a7a1-e4886fd9b672 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:03.814740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a7303-f4bf-4677-914d-29258aad9382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:03.815383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a7303-f4bf-4677-914d-29258aad9382 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:03.815663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a7303-f4bf-4677-914d-29258aad9382 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:03.815935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a7303-f4bf-4677-914d-29258aad9382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:03.817901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-633a7303-f4bf-4677-914d-29258aad9382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:03.819224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-736b5eab-a98f-4793-b887-1e5e47dc43c9 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:03.820445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-736b5eab-a98f-4793-b887-1e5e47dc43c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:03.832303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1094/2203] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:32:03.837383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b5eab-a98f-4793-b887-1e5e47dc43c9 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:03.851295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b5eab-a98f-4793-b887-1e5e47dc43c9 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:03.866966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4f6ca-78da-4be1-8b01-f2cc69eb5b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:03.868918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4f6ca-78da-4be1-8b01-f2cc69eb5b98 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:03.869393 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4f6ca-78da-4be1-8b01-f2cc69eb5b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:03.869678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4f6ca-78da-4be1-8b01-f2cc69eb5b98 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:03.870409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4f6ca-78da-4be1-8b01-f2cc69eb5b98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:32:03.871787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4f6ca-78da-4be1-8b01-f2cc69eb5b98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:03.873492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4f6ca-78da-4be1-8b01-f2cc69eb5b98 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:03.875420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:32:03.892539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1095/2204] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:32:03.893802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4f6ca-78da-4be1-8b01-f2cc69eb5b98 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:03.907809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4f6ca-78da-4be1-8b01-f2cc69eb5b98 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:03.918862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5bb33-c3b8-4ee2-ab29-48baebdb5643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:03.919547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5bb33-c3b8-4ee2-ab29-48baebdb5643 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:03.919826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5bb33-c3b8-4ee2-ab29-48baebdb5643 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:03.920147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5bb33-c3b8-4ee2-ab29-48baebdb5643 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:03.920826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5bb33-c3b8-4ee2-ab29-48baebdb5643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:03.923039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5bb33-c3b8-4ee2-ab29-48baebdb5643 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:03.923241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a7303-f4bf-4677-914d-29258aad9382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:03.925483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a7303-f4bf-4677-914d-29258aad9382 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:03.939500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1110/2205] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:32:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:32:03.968769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1096/2206] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:32:03 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:32:03.974338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a7303-f4bf-4677-914d-29258aad9382 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:03.989356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a7303-f4bf-4677-914d-29258aad9382 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:04.085010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b68b1-f544-4d7c-8ab6-5cd1ce817dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:04.085632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b68b1-f544-4d7c-8ab6-5cd1ce817dfa admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:04.085909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b68b1-f544-4d7c-8ab6-5cd1ce817dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:04.086180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b68b1-f544-4d7c-8ab6-5cd1ce817dfa admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:04.086961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b68b1-f544-4d7c-8ab6-5cd1ce817dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:04.088932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b68b1-f544-4d7c-8ab6-5cd1ce817dfa admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:04.116304 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1111/2207] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:32:03 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:32:04.120024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5bb33-c3b8-4ee2-ab29-48baebdb5643 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:04.166744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5bb33-c3b8-4ee2-ab29-48baebdb5643 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:04.179936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69266085-e0e7-4a26-97e5-2323117833dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:04.180870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69266085-e0e7-4a26-97e5-2323117833dc admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:04.181127 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69266085-e0e7-4a26-97e5-2323117833dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:04.181490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69266085-e0e7-4a26-97e5-2323117833dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:04.182392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69266085-e0e7-4a26-97e5-2323117833dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:04.186296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69266085-e0e7-4a26-97e5-2323117833dc admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:04.203221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1097/2208] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:32:04.207487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b68b1-f544-4d7c-8ab6-5cd1ce817dfa admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:04.221280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b68b1-f544-4d7c-8ab6-5cd1ce817dfa admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:04.226167 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c932d-31e6-4c43-b27c-4919df40c6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:04.226773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c932d-31e6-4c43-b27c-4919df40c6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:04.227067 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c932d-31e6-4c43-b27c-4919df40c6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:04.227331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c932d-31e6-4c43-b27c-4919df40c6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:04.232553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c932d-31e6-4c43-b27c-4919df40c6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:04.232553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c932d-31e6-4c43-b27c-4919df40c6f0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:04.248341 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1112/2209] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:32:04.250100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69266085-e0e7-4a26-97e5-2323117833dc admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:04.268851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69266085-e0e7-4a26-97e5-2323117833dc admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:04.275486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0199f-c236-4186-aa97-87b952d8398a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:04.276374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0199f-c236-4186-aa97-87b952d8398a admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:04.277387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0199f-c236-4186-aa97-87b952d8398a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:04.277803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0199f-c236-4186-aa97-87b952d8398a admin admin] PATH_INFO: `/v3/projects/3747df9616604e22826b8f308eac57e3/users/04edb638af8c43a78db9ef629154df83/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:04.301474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0199f-c236-4186-aa97-87b952d8398a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3747df9616604e22826b8f308eac57e3, user_id=04edb638af8c43a78db9ef629154df83, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:04.304533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0199f-c236-4186-aa97-87b952d8398a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:04.322086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1098/2210] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:32:04 2022] PUT /identity/v3/projects/3747df9616604e22826b8f308eac57e3/users/04edb638af8c43a78db9ef629154df83/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:32:04.333046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf93d-b2d7-4b5c-a2b1-03f24fdc572e None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:04.333344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf93d-b2d7-4b5c-a2b1-03f24fdc572e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:04.333631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf93d-b2d7-4b5c-a2b1-03f24fdc572e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:04.347384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c55506-8bae-449c-bf09-3b5f29675571 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:04.347808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c55506-8bae-449c-bf09-3b5f29675571 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:04.348016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c55506-8bae-449c-bf09-3b5f29675571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:04.370082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-14c55506-8bae-449c-bf09-3b5f29675571 None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:04.371840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-14c55506-8bae-449c-bf09-3b5f29675571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:04.373652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-38ecf93d-b2d7-4b5c-a2b1-03f24fdc572e None None] MFA Rules not processed for user `04edb638af8c43a78db9ef629154df83`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:04.374867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-38ecf93d-b2d7-4b5c-a2b1-03f24fdc572e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:04.431064 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:32:04.445912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1099/2211] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:32:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:32:04.482043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c55506-8bae-449c-bf09-3b5f29675571 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:04.484570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1113/2212] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:32:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:32:04.491328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-14c55506-8bae-449c-bf09-3b5f29675571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:04.498918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf93d-b2d7-4b5c-a2b1-03f24fdc572e None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:04.500541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-38ecf93d-b2d7-4b5c-a2b1-03f24fdc572e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:04.606124 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf93d-b2d7-4b5c-a2b1-03f24fdc572e None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:04.610902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefb995-193d-4ce5-b1db-7aa61132ce93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:04.611763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefb995-193d-4ce5-b1db-7aa61132ce93 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:04.612054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefb995-193d-4ce5-b1db-7aa61132ce93 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:04.612332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefb995-193d-4ce5-b1db-7aa61132ce93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:04.624440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c55506-8bae-449c-bf09-3b5f29675571 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:04.626762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefb995-193d-4ce5-b1db-7aa61132ce93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:04.628751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefb995-193d-4ce5-b1db-7aa61132ce93 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:04.630782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d52f6-9005-490d-b7d7-9b26e1564d26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:04.631483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d52f6-9005-490d-b7d7-9b26e1564d26 service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:04.631824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d52f6-9005-490d-b7d7-9b26e1564d26 service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:04.632161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d52f6-9005-490d-b7d7-9b26e1564d26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:04.643752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d52f6-9005-490d-b7d7-9b26e1564d26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:04.646975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d52f6-9005-490d-b7d7-9b26e1564d26 service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:04.657999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1114/2213] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:32:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:32:04.660089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1100/2214] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:32:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:32:06.317642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814ae6f-bfdf-4158-a6ea-41fdb01b9977 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:06.318154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814ae6f-bfdf-4158-a6ea-41fdb01b9977 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:06.318154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814ae6f-bfdf-4158-a6ea-41fdb01b9977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:06.342571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-1814ae6f-bfdf-4158-a6ea-41fdb01b9977 None None] MFA Rules not processed for user `04edb638af8c43a78db9ef629154df83`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:06.344031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1814ae6f-bfdf-4158-a6ea-41fdb01b9977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:06.359241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1115/2215] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:32:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:14.655347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:32:16.336129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=38 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:32:21.169974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964efad9-29d5-4d8e-9b76-23bf76e83c56 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:21.170540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964efad9-29d5-4d8e-9b76-23bf76e83c56 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:21.170540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964efad9-29d5-4d8e-9b76-23bf76e83c56 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:21.171876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1101/2216] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:32:21.177503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec2b8b-4289-4fb3-95cd-bd5414d4bc62 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:21.177935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec2b8b-4289-4fb3-95cd-bd5414d4bc62 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:21.178047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec2b8b-4289-4fb3-95cd-bd5414d4bc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:21.199399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-21ec2b8b-4289-4fb3-95cd-bd5414d4bc62 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:21.200678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-21ec2b8b-4289-4fb3-95cd-bd5414d4bc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:21.300900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1116/2217] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:32:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:21.314253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964efad9-29d5-4d8e-9b76-23bf76e83c56 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:21.316861 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-964efad9-29d5-4d8e-9b76-23bf76e83c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:21.392793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964efad9-29d5-4d8e-9b76-23bf76e83c56 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:21.398309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33899cbf-7475-4b89-9c84-71a3bc5d37e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52470b620abd4cd29ab7e4752f285bb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:21.399069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33899cbf-7475-4b89-9c84-71a3bc5d37e7 service glance] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:21.399445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33899cbf-7475-4b89-9c84-71a3bc5d37e7 service glance] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:21.399829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33899cbf-7475-4b89-9c84-71a3bc5d37e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:21.411317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33899cbf-7475-4b89-9c84-71a3bc5d37e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:21.412990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33899cbf-7475-4b89-9c84-71a3bc5d37e7 service glance] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:21.425096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1102/2218] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:32:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:32:21.496424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526156e-ed44-4469-a214-042bae097b95 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:21.496746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526156e-ed44-4469-a214-042bae097b95 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:21.497026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526156e-ed44-4469-a214-042bae097b95 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:21.498389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1117/2219] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:32:21.503534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dac50-2329-4cb1-9f9b-78b8f49ac0ff None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:21.503814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dac50-2329-4cb1-9f9b-78b8f49ac0ff None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:21.504082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dac50-2329-4cb1-9f9b-78b8f49ac0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:21.523995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-972dac50-2329-4cb1-9f9b-78b8f49ac0ff None None] MFA Rules not processed for user `cdf97e9e0340441c87316ce03d46cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:21.525261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-972dac50-2329-4cb1-9f9b-78b8f49ac0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:21.575234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1103/2220] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:32:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:21.587235 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526156e-ed44-4469-a214-042bae097b95 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:21.589664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8526156e-ed44-4469-a214-042bae097b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:21.602616 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526156e-ed44-4469-a214-042bae097b95 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:21.606804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf0d88-a482-481a-b076-8371ebf35e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:21.607388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf0d88-a482-481a-b076-8371ebf35e3e service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:21.607661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf0d88-a482-481a-b076-8371ebf35e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:21.607957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf0d88-a482-481a-b076-8371ebf35e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:21.614799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf0d88-a482-481a-b076-8371ebf35e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:21.616222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf0d88-a482-481a-b076-8371ebf35e3e service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:21.624194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1118/2221] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:32:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:32:21.708013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04f942-de3d-43cf-b029-6870cd6885ac None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:21.708318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04f942-de3d-43cf-b029-6870cd6885ac None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:21.708629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04f942-de3d-43cf-b029-6870cd6885ac None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:21.709994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1104/2222] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:32:21.716804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aff787-5516-4112-8d50-2cb5c8ff5c45 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:21.717087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aff787-5516-4112-8d50-2cb5c8ff5c45 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:21.717385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aff787-5516-4112-8d50-2cb5c8ff5c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:21.737961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d5aff787-5516-4112-8d50-2cb5c8ff5c45 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:21.739165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d5aff787-5516-4112-8d50-2cb5c8ff5c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:21.748698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1119/2223] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:32:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:21.758940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04f942-de3d-43cf-b029-6870cd6885ac None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:21.770030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04f942-de3d-43cf-b029-6870cd6885ac None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:21.773690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302a74b-6bee-4e1f-af34-3a719c5aca24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:21.774324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302a74b-6bee-4e1f-af34-3a719c5aca24 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:21.774600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302a74b-6bee-4e1f-af34-3a719c5aca24 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:21.774870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302a74b-6bee-4e1f-af34-3a719c5aca24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:21.789356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302a74b-6bee-4e1f-af34-3a719c5aca24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:21.790947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302a74b-6bee-4e1f-af34-3a719c5aca24 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:21.804955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1105/2224] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:32:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:32:26.703665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746d993-9217-4ec7-a97e-2b90fab479ef None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:26.704149 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746d993-9217-4ec7-a97e-2b90fab479ef None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:26.704237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746d993-9217-4ec7-a97e-2b90fab479ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:26.723112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0746d993-9217-4ec7-a97e-2b90fab479ef None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:26.724307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0746d993-9217-4ec7-a97e-2b90fab479ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:26.732048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1120/2225] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:32:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:26.744019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302a74b-6bee-4e1f-af34-3a719c5aca24 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:26.756319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302a74b-6bee-4e1f-af34-3a719c5aca24 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:26.761284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2689f69-60df-4973-a0fc-da701308668c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:26.761874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2689f69-60df-4973-a0fc-da701308668c service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:26.762149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2689f69-60df-4973-a0fc-da701308668c service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:26.762415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2689f69-60df-4973-a0fc-da701308668c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:26.770279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2689f69-60df-4973-a0fc-da701308668c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:26.771884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2689f69-60df-4973-a0fc-da701308668c service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:26.780459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1106/2226] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:32:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:32:27.746977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47734f48-8ddb-4a1e-b36e-128fd7fb655d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:27.747856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47734f48-8ddb-4a1e-b36e-128fd7fb655d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:27.748140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47734f48-8ddb-4a1e-b36e-128fd7fb655d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:27.768824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-47734f48-8ddb-4a1e-b36e-128fd7fb655d None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:27.770300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-47734f48-8ddb-4a1e-b36e-128fd7fb655d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:27.778492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1121/2227] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:32:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:27.790791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2689f69-60df-4973-a0fc-da701308668c service swift] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:27.802271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2689f69-60df-4973-a0fc-da701308668c service swift] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:27.809313 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484e53f-d176-430e-84b3-72bdd617b85b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:27.809942 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484e53f-d176-430e-84b3-72bdd617b85b service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:27.810220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484e53f-d176-430e-84b3-72bdd617b85b service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:27.810494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484e53f-d176-430e-84b3-72bdd617b85b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:27.820075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484e53f-d176-430e-84b3-72bdd617b85b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:27.821816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484e53f-d176-430e-84b3-72bdd617b85b service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:27.834517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1107/2228] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:32:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:32:28.449799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc63c5-a08f-4aab-acd5-70b636127a5f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:28.450269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc63c5-a08f-4aab-acd5-70b636127a5f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:28.450654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc63c5-a08f-4aab-acd5-70b636127a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:28.473404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-c1bc63c5-a08f-4aab-acd5-70b636127a5f None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:28.474648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c1bc63c5-a08f-4aab-acd5-70b636127a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:28.483462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1122/2229] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 00:32:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:28.506782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484e53f-d176-430e-84b3-72bdd617b85b service swift] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:28.521850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484e53f-d176-430e-84b3-72bdd617b85b service swift] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:28.528673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1de97d-71b6-4b57-a7eb-f6b30479ba75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:28.529427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1de97d-71b6-4b57-a7eb-f6b30479ba75 service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:28.529791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1de97d-71b6-4b57-a7eb-f6b30479ba75 service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:28.530072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1de97d-71b6-4b57-a7eb-f6b30479ba75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:28.539035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1de97d-71b6-4b57-a7eb-f6b30479ba75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:28.543212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1de97d-71b6-4b57-a7eb-f6b30479ba75 service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:28.550226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1108/2230] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:32:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:32:28.799692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c57114-813f-4602-99f1-1cedb22b40b1 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:28.800199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c57114-813f-4602-99f1-1cedb22b40b1 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:28.800307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c57114-813f-4602-99f1-1cedb22b40b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:28.823206 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-02c57114-813f-4602-99f1-1cedb22b40b1 None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:28.824472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-02c57114-813f-4602-99f1-1cedb22b40b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:28.832003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1123/2231] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:32:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:28.851160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1de97d-71b6-4b57-a7eb-f6b30479ba75 service swift] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:28.863256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1de97d-71b6-4b57-a7eb-f6b30479ba75 service swift] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:28.868850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ae9b9-dba4-428f-80b8-fb4a520ad87e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:28.870519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ae9b9-dba4-428f-80b8-fb4a520ad87e service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:28.870909 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ae9b9-dba4-428f-80b8-fb4a520ad87e service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:28.871229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ae9b9-dba4-428f-80b8-fb4a520ad87e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:28.880374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ae9b9-dba4-428f-80b8-fb4a520ad87e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:28.881979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ae9b9-dba4-428f-80b8-fb4a520ad87e service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:28.890998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1109/2232] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:32:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:32:29.305199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40a0c9-96b0-46fc-81b2-87b851a247e5 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:29.306110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40a0c9-96b0-46fc-81b2-87b851a247e5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:29.307511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40a0c9-96b0-46fc-81b2-87b851a247e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:29.336484 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5c40a0c9-96b0-46fc-81b2-87b851a247e5 None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:29.338406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5c40a0c9-96b0-46fc-81b2-87b851a247e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:29.346615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1124/2233] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 00:32:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:29.361577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ae9b9-dba4-428f-80b8-fb4a520ad87e service swift] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:29.385142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ae9b9-dba4-428f-80b8-fb4a520ad87e service swift] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:29.390935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47a4c4-5eb9-41c7-b678-22906acf1b00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:29.391557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47a4c4-5eb9-41c7-b678-22906acf1b00 service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:29.391838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47a4c4-5eb9-41c7-b678-22906acf1b00 service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:29.392114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47a4c4-5eb9-41c7-b678-22906acf1b00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:29.402445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47a4c4-5eb9-41c7-b678-22906acf1b00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:29.404052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47a4c4-5eb9-41c7-b678-22906acf1b00 service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:29.412938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1110/2234] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:32:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:32:31.810125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40a0c9-96b0-46fc-81b2-87b851a247e5 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:31.880760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40a0c9-96b0-46fc-81b2-87b851a247e5 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:31.885885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:31.886521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:31.886804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:31.887082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] PATH_INFO: `/v3/users/04edb638af8c43a78db9ef629154df83` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:31.888485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] RBAC: Authorizing `identity:delete_user(user_id=04edb638af8c43a78db9ef629154df83)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:31.890554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:31.942366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:32:31.949516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04edb638af8c43a78db9ef629154df83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:31.950610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04edb638af8c43a78db9ef629154df83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:31.959482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04edb638af8c43a78db9ef629154df83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:31.967481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04edb638af8c43a78db9ef629154df83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:31.968199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04edb638af8c43a78db9ef629154df83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:31.987714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1125/2235] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:32:31 2022] DELETE /identity/v3/users/04edb638af8c43a78db9ef629154df83 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:32:31.992403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8e7d3-900a-4b2c-816a-a3a63a7591ba None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:31.993906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8e7d3-900a-4b2c-816a-a3a63a7591ba None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:31.994915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8e7d3-900a-4b2c-816a-a3a63a7591ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:32.015683 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-d1d8e7d3-900a-4b2c-816a-a3a63a7591ba None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:32.017058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d1d8e7d3-900a-4b2c-816a-a3a63a7591ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:32.048972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:32:32.082631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1111/2236] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:32.092749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:32.094411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:32.155985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7232e1-f269-4f7f-ab7f-dad4b47f666b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:32.159939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f3dd4-10f9-4778-8b9e-67ace4fe9b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:32.160595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f3dd4-10f9-4778-8b9e-67ace4fe9b0b service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:32.160879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f3dd4-10f9-4778-8b9e-67ace4fe9b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:32.161153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f3dd4-10f9-4778-8b9e-67ace4fe9b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:32.172745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f3dd4-10f9-4778-8b9e-67ace4fe9b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:32.174668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f3dd4-10f9-4778-8b9e-67ace4fe9b0b service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:32.188278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1126/2237] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:32:32.432315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8e7d3-900a-4b2c-816a-a3a63a7591ba None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:32.433969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d1d8e7d3-900a-4b2c-816a-a3a63a7591ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:32.450501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8e7d3-900a-4b2c-816a-a3a63a7591ba None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:32.455998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:32.456854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:32.457142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:32.457415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] PATH_INFO: `/v3/projects/3747df9616604e22826b8f308eac57e3` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:32.458678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=3747df9616604e22826b8f308eac57e3)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:32.461090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:32.535330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3747df9616604e22826b8f308eac57e3', 'request_id': 'req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:32.542982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3747df9616604e22826b8f308eac57e3', 'request_id': 'req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:32.553573 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3747df9616604e22826b8f308eac57e3', 'request_id': 'req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:32.569359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] The token cache is being invalidate because project 3747df9616604e22826b8f308eac57e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:32:32.569687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:32.570396 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-36d9f49a-23ff-4cc2-a008-1a12a84b71ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:32:32.576542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1112/2238] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:32:32 2022] DELETE /identity/v3/projects/3747df9616604e22826b8f308eac57e3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:32:42.181788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=40,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:32:42.565626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:32:48.168109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9047b-77a6-4017-804b-979aa872514e None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:48.168622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9047b-77a6-4017-804b-979aa872514e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:48.168737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9047b-77a6-4017-804b-979aa872514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:48.187876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-15f9047b-77a6-4017-804b-979aa872514e None None] MFA Rules not processed for user `e7a36e4630904f4eac08c9de1b8235ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:48.189092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-15f9047b-77a6-4017-804b-979aa872514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:48.246527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1127/2239] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:32:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:52.429973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c78294-f614-40a0-8932-e14bdde440ec None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:52.430983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c78294-f614-40a0-8932-e14bdde440ec None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:52.431329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c78294-f614-40a0-8932-e14bdde440ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:52.451663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-d2c78294-f614-40a0-8932-e14bdde440ec None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:32:52.452920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d2c78294-f614-40a0-8932-e14bdde440ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:52.507987 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1113/2240] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 00:32:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:32:52.531571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9047b-77a6-4017-804b-979aa872514e None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:52.533312 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-15f9047b-77a6-4017-804b-979aa872514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:52.628888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9047b-77a6-4017-804b-979aa872514e None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:52.636892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca12f1-302e-45c7-99ea-7f0705b84cf3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:52.637208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca12f1-302e-45c7-99ea-7f0705b84cf3 service swift] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:52.637540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca12f1-302e-45c7-99ea-7f0705b84cf3 service swift] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:52.637882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca12f1-302e-45c7-99ea-7f0705b84cf3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:52.647749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca12f1-302e-45c7-99ea-7f0705b84cf3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:52.649807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca12f1-302e-45c7-99ea-7f0705b84cf3 service swift] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:52.658651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1128/2241] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:32:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:32:55.609544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c78294-f614-40a0-8932-e14bdde440ec None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:32:55.610978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d2c78294-f614-40a0-8932-e14bdde440ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:55.686417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c78294-f614-40a0-8932-e14bdde440ec None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:32:55.691311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59874a0e-3366-4705-a0ef-7b239aab4887 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:32:55.691921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59874a0e-3366-4705-a0ef-7b239aab4887 service placement] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:32:55.692196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59874a0e-3366-4705-a0ef-7b239aab4887 service placement] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:32:55.692462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59874a0e-3366-4705-a0ef-7b239aab4887 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:32:55.694204 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-59874a0e-3366-4705-a0ef-7b239aab4887 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:32:55.706633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59874a0e-3366-4705-a0ef-7b239aab4887 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:32:55.708242 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59874a0e-3366-4705-a0ef-7b239aab4887 service placement] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:32:55.718464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1114/2242] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:32:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:33:02.655194 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=57 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:33:05.714214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=57 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:33:41.796476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af40e7-5b9a-42db-ba5b-d47da2e36520 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:33:41.797063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af40e7-5b9a-42db-ba5b-d47da2e36520 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:33:41.797168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af40e7-5b9a-42db-ba5b-d47da2e36520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:33:41.817256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-37af40e7-5b9a-42db-ba5b-d47da2e36520 None None] MFA Rules not processed for user `a6f08eaef2d149a9b0236d4bcbe03a92`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:33:41.818497 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-37af40e7-5b9a-42db-ba5b-d47da2e36520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:33:41.877562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1129/2243] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:33:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:33:42.015391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a7888-48f6-4d72-9088-fa193c015db0 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:33:42.015901 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a7888-48f6-4d72-9088-fa193c015db0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:33:42.015996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a7888-48f6-4d72-9088-fa193c015db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:33:42.035195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-eb9a7888-48f6-4d72-9088-fa193c015db0 None None] MFA Rules not processed for user `ecf29db866eb43fa954b0d440577f45f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:33:42.036379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-eb9a7888-48f6-4d72-9088-fa193c015db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:33:42.097967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1115/2244] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:33:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:33:51.870499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=25 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:33:52.090225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=25 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:00.534400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfa2a8-e8e9-444d-862e-715061d6efbe None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:00.534910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfa2a8-e8e9-444d-862e-715061d6efbe None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:00.535006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfa2a8-e8e9-444d-862e-715061d6efbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:00.556028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d0bfa2a8-e8e9-444d-862e-715061d6efbe None None] MFA Rules not processed for user `e7a36e4630904f4eac08c9de1b8235ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:00.557298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d0bfa2a8-e8e9-444d-862e-715061d6efbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:00.564748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1130/2245] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:07.022271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002da12-006c-4bea-9eff-d7c251d65775 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:07.022832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002da12-006c-4bea-9eff-d7c251d65775 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:07.022832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002da12-006c-4bea-9eff-d7c251d65775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:07.042039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0002da12-006c-4bea-9eff-d7c251d65775 None None] MFA Rules not processed for user `5bf265cb4af14e2d8815c42288e2a1f7`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:07.043253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0002da12-006c-4bea-9eff-d7c251d65775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:07.099394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1116/2246] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:10.550110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:17.092463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=25 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:43.461689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfa2a8-e8e9-444d-862e-715061d6efbe None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:43.463227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d0bfa2a8-e8e9-444d-862e-715061d6efbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:43.520384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bfa2a8-e8e9-444d-862e-715061d6efbe None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:43.523990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a04d0d-018c-45a3-bfa0-3dc519c689c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:43.524654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a04d0d-018c-45a3-bfa0-3dc519c689c2 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:43.524932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a04d0d-018c-45a3-bfa0-3dc519c689c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:43.525208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a04d0d-018c-45a3-bfa0-3dc519c689c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:43.526904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a5a04d0d-018c-45a3-bfa0-3dc519c689c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:43.588856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a04d0d-018c-45a3-bfa0-3dc519c689c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:43.590236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a04d0d-018c-45a3-bfa0-3dc519c689c2 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:43.600225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1131/2247] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:34:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:34:44.603286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffac5cc-e474-4d45-922c-f9c32d405876 None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:44.603859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffac5cc-e474-4d45-922c-f9c32d405876 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:44.604131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffac5cc-e474-4d45-922c-f9c32d405876 None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:44.606221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1117/2248] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:34:44.612355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7724-aa71-47df-9f8d-50693e80232e None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:44.613294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7724-aa71-47df-9f8d-50693e80232e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:44.614543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca7724-aa71-47df-9f8d-50693e80232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:44.635644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-8aca7724-aa71-47df-9f8d-50693e80232e None None] MFA Rules not processed for user `cdf97e9e0340441c87316ce03d46cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:44.636928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8aca7724-aa71-47df-9f8d-50693e80232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:44.643380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1132/2249] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:34:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:34:44.662865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffac5cc-e474-4d45-922c-f9c32d405876 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:44.676433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffac5cc-e474-4d45-922c-f9c32d405876 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:44.680549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bb3de7-0f2b-4ea7-bf5f-23054dd485d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:44.681164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb3de7-0f2b-4ea7-bf5f-23054dd485d1 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:44.681442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb3de7-0f2b-4ea7-bf5f-23054dd485d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:44.681711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bb3de7-0f2b-4ea7-bf5f-23054dd485d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:44.689539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb3de7-0f2b-4ea7-bf5f-23054dd485d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:44.691010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bb3de7-0f2b-4ea7-bf5f-23054dd485d1 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:44.699949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1118/2250] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:34:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:34:44.829097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f58856-e731-4437-a453-72b0477bba48 None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:44.829376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f58856-e731-4437-a453-72b0477bba48 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:44.829670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f58856-e731-4437-a453-72b0477bba48 None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:44.831713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1133/2251] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:34:44.836636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ead871-b179-441e-bb71-a46aae76a52a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:44.836927 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ead871-b179-441e-bb71-a46aae76a52a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:44.837203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ead871-b179-441e-bb71-a46aae76a52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:44.859615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-40ead871-b179-441e-bb71-a46aae76a52a None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:44.860905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-40ead871-b179-441e-bb71-a46aae76a52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:44.940527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1119/2252] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:34:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:44.951765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f58856-e731-4437-a453-72b0477bba48 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:44.954235 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a1f58856-e731-4437-a453-72b0477bba48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:44.969221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f58856-e731-4437-a453-72b0477bba48 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:44.974596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef7b0b-66e5-4753-89f4-ff2f31c17138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:44.975348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef7b0b-66e5-4753-89f4-ff2f31c17138 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:44.975720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef7b0b-66e5-4753-89f4-ff2f31c17138 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:44.976086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef7b0b-66e5-4753-89f4-ff2f31c17138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:44.990513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef7b0b-66e5-4753-89f4-ff2f31c17138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:44.992151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef7b0b-66e5-4753-89f4-ff2f31c17138 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:45.008581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1134/2253] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:34:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:34:45.809247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ead871-b179-441e-bb71-a46aae76a52a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:45.819837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ead871-b179-441e-bb71-a46aae76a52a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:45.824078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:45.824698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:45.825206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:45.825440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:45.828163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:45.844985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:45.846457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:45.863391 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1120/2254] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:34:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:34:46.476191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef7b0b-66e5-4753-89f4-ff2f31c17138 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:46.485434 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef7b0b-66e5-4753-89f4-ff2f31c17138 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:46.490038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4509e-7262-4c8f-8b42-175c226a7183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:46.490714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4509e-7262-4c8f-8b42-175c226a7183 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:46.491058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4509e-7262-4c8f-8b42-175c226a7183 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:46.491396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4509e-7262-4c8f-8b42-175c226a7183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:46.493231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b6e4509e-7262-4c8f-8b42-175c226a7183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:46.507998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4509e-7262-4c8f-8b42-175c226a7183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:46.509596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4509e-7262-4c8f-8b42-175c226a7183 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:46.521774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1135/2255] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:34:47.499589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:47.501066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:47.514265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65614842-f6c7-4808-8b2f-54c17316a01e service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:47.521079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:47.521820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:47.522029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:47.522237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] PATH_INFO: `/v3/users/a6f08eaef2d149a9b0236d4bcbe03a92` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:47.523412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f08eaef2d149a9b0236d4bcbe03a92)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:47.525373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:47.576784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f08eaef2d149a9b0236d4bcbe03a92'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:47.577709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f08eaef2d149a9b0236d4bcbe03a92'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:47.590787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f08eaef2d149a9b0236d4bcbe03a92'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:47.598559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f08eaef2d149a9b0236d4bcbe03a92'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:47.607043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-93838f84-5f01-4465-8458-408638548a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f08eaef2d149a9b0236d4bcbe03a92'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:47.613501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1121/2256] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:34:47 2022] DELETE /identity/v3/users/a6f08eaef2d149a9b0236d4bcbe03a92 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:47.865819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4509e-7262-4c8f-8b42-175c226a7183 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:47.883352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4509e-7262-4c8f-8b42-175c226a7183 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:47.940795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:47.941426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:47.941713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:47.942127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] PATH_INFO: `/v3/projects/64e86923fd7b42cfaf18408210d4f6d7` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:47.943255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e86923fd7b42cfaf18408210d4f6d7)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:47.945227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:47.965861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:47.991225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e86923fd7b42cfaf18408210d4f6d7', 'request_id': 'req-e4774781-9d0d-49fc-96b8-a2442999e7f3'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:47.996568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e86923fd7b42cfaf18408210d4f6d7', 'request_id': 'req-e4774781-9d0d-49fc-96b8-a2442999e7f3'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:48.002527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e86923fd7b42cfaf18408210d4f6d7', 'request_id': 'req-e4774781-9d0d-49fc-96b8-a2442999e7f3'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:48.014382 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] The token cache is being invalidate because project 64e86923fd7b42cfaf18408210d4f6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:34:48.014670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:48.015307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e4774781-9d0d-49fc-96b8-a2442999e7f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:48.016819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1136/2257] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:34:47 2022] DELETE /identity/v3/projects/64e86923fd7b42cfaf18408210d4f6d7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:48.021974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d584f8-a205-4890-b94f-72930e89aadc None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.022255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d584f8-a205-4890-b94f-72930e89aadc None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.022526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d584f8-a205-4890-b94f-72930e89aadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.042312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-c6d584f8-a205-4890-b94f-72930e89aadc None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:48.043509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-c6d584f8-a205-4890-b94f-72930e89aadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:48.075950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:48.104965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1122/2258] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:48.112691 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ddef-afd8-43b7-9ab7-45a18eb385cf None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.112974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ddef-afd8-43b7-9ab7-45a18eb385cf None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.113248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132ddef-afd8-43b7-9ab7-45a18eb385cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.133541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b132ddef-afd8-43b7-9ab7-45a18eb385cf None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:48.134740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b132ddef-afd8-43b7-9ab7-45a18eb385cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:48.142476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1137/2259] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:48.147235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d584f8-a205-4890-b94f-72930e89aadc None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.156818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d584f8-a205-4890-b94f-72930e89aadc None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.161347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cda11-665f-4c5f-87c7-f580ed2632d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.161963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cda11-665f-4c5f-87c7-f580ed2632d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.162238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cda11-665f-4c5f-87c7-f580ed2632d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.162503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cda11-665f-4c5f-87c7-f580ed2632d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.163138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cda11-665f-4c5f-87c7-f580ed2632d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:34:48.163434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cda11-665f-4c5f-87c7-f580ed2632d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.165066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cda11-665f-4c5f-87c7-f580ed2632d5 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.176903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1123/2260] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:34:48.181190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132ddef-afd8-43b7-9ab7-45a18eb385cf None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.190507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132ddef-afd8-43b7-9ab7-45a18eb385cf None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.196103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176ae62-e44e-477f-9d39-92859c2cdabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.196712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176ae62-e44e-477f-9d39-92859c2cdabe admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.196985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176ae62-e44e-477f-9d39-92859c2cdabe admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.197251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176ae62-e44e-477f-9d39-92859c2cdabe admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.197869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176ae62-e44e-477f-9d39-92859c2cdabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.199746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176ae62-e44e-477f-9d39-92859c2cdabe admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.227009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1138/2261] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:34:48 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:34:48.231472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cda11-665f-4c5f-87c7-f580ed2632d5 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.242312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cda11-665f-4c5f-87c7-f580ed2632d5 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.297728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766146cf-1dd5-4767-a12e-9d3c4400fd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.298335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766146cf-1dd5-4767-a12e-9d3c4400fd38 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.298611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766146cf-1dd5-4767-a12e-9d3c4400fd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.298882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766146cf-1dd5-4767-a12e-9d3c4400fd38 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.299513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766146cf-1dd5-4767-a12e-9d3c4400fd38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.302125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766146cf-1dd5-4767-a12e-9d3c4400fd38 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.323864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1124/2262] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:34:48 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:34:48.328367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176ae62-e44e-477f-9d39-92859c2cdabe admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.338589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176ae62-e44e-477f-9d39-92859c2cdabe admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.343151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdb3dd-9261-4863-bfc7-a507cda0c81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.343749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdb3dd-9261-4863-bfc7-a507cda0c81b admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.344029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdb3dd-9261-4863-bfc7-a507cda0c81b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.344302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdb3dd-9261-4863-bfc7-a507cda0c81b admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.344978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdb3dd-9261-4863-bfc7-a507cda0c81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.346591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdb3dd-9261-4863-bfc7-a507cda0c81b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.356811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1139/2263] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:34:48.360717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766146cf-1dd5-4767-a12e-9d3c4400fd38 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.371655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766146cf-1dd5-4767-a12e-9d3c4400fd38 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.376150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3d6c7-62ad-4107-bad5-72cdd7bc3a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.376770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3d6c7-62ad-4107-bad5-72cdd7bc3a97 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.377046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3d6c7-62ad-4107-bad5-72cdd7bc3a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.377314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3d6c7-62ad-4107-bad5-72cdd7bc3a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.377945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3d6c7-62ad-4107-bad5-72cdd7bc3a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.380143 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3d6c7-62ad-4107-bad5-72cdd7bc3a97 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.390647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1125/2264] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:34:48.395367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdb3dd-9261-4863-bfc7-a507cda0c81b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.405369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdb3dd-9261-4863-bfc7-a507cda0c81b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.411146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b6469-5d58-4790-8fc4-316b28f2946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.411802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b6469-5d58-4790-8fc4-316b28f2946b admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.412075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b6469-5d58-4790-8fc4-316b28f2946b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.412347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b6469-5d58-4790-8fc4-316b28f2946b admin admin] PATH_INFO: `/v3/projects/877691afa51e47bb8891a4de1c7c5cd2/users/e7e3cc07a8cc42e9bc2cbf71586d39bb/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.431383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b6469-5d58-4790-8fc4-316b28f2946b admin admin] RBAC: Authorizing `identity:create_grant(project_id=877691afa51e47bb8891a4de1c7c5cd2, user_id=e7e3cc07a8cc42e9bc2cbf71586d39bb, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.434398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b6469-5d58-4790-8fc4-316b28f2946b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.449407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1140/2265] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:34:48 2022] PUT /identity/v3/projects/877691afa51e47bb8891a4de1c7c5cd2/users/e7e3cc07a8cc42e9bc2cbf71586d39bb/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:48.460483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65494fb-5d5d-4ccc-8dbd-e3082264ed4b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.460795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65494fb-5d5d-4ccc-8dbd-e3082264ed4b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.461068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65494fb-5d5d-4ccc-8dbd-e3082264ed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.491644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b65494fb-5d5d-4ccc-8dbd-e3082264ed4b None None] MFA Rules not processed for user `e7e3cc07a8cc42e9bc2cbf71586d39bb`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:48.493019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b65494fb-5d5d-4ccc-8dbd-e3082264ed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:48.558720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1126/2266] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:48.563260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b6469-5d58-4790-8fc4-316b28f2946b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.626491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b6469-5d58-4790-8fc4-316b28f2946b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.631487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c1e79-8948-463a-8301-d282a8b272b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.632093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c1e79-8948-463a-8301-d282a8b272b7 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.632371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c1e79-8948-463a-8301-d282a8b272b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.632721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c1e79-8948-463a-8301-d282a8b272b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.633331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c1e79-8948-463a-8301-d282a8b272b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.635302 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c1e79-8948-463a-8301-d282a8b272b7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.666816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1141/2267] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:34:48 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:34:48.670923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65494fb-5d5d-4ccc-8dbd-e3082264ed4b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.682602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65494fb-5d5d-4ccc-8dbd-e3082264ed4b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.717050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:48.746360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36b57b-88fe-426c-8c53-f9d92e90658c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.746884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36b57b-88fe-426c-8c53-f9d92e90658c admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.747160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36b57b-88fe-426c-8c53-f9d92e90658c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.747433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36b57b-88fe-426c-8c53-f9d92e90658c admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.748055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36b57b-88fe-426c-8c53-f9d92e90658c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.750027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36b57b-88fe-426c-8c53-f9d92e90658c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.771020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1127/2268] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:34:48 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:34:48.775032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c1e79-8948-463a-8301-d282a8b272b7 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.786045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c1e79-8948-463a-8301-d282a8b272b7 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.790928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5074a-c572-4112-9f61-65d8e11b3faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.791525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5074a-c572-4112-9f61-65d8e11b3faa admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.791804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5074a-c572-4112-9f61-65d8e11b3faa admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.792075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5074a-c572-4112-9f61-65d8e11b3faa admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.792772 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5074a-c572-4112-9f61-65d8e11b3faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.794481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5074a-c572-4112-9f61-65d8e11b3faa admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.805727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1142/2269] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:34:48.809768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36b57b-88fe-426c-8c53-f9d92e90658c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.819698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36b57b-88fe-426c-8c53-f9d92e90658c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:48.824289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9f53a-130e-4887-ae66-324e596af65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:48.824970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9f53a-130e-4887-ae66-324e596af65b admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:48.825246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9f53a-130e-4887-ae66-324e596af65b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:48.825515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9f53a-130e-4887-ae66-324e596af65b admin admin] PATH_INFO: `/v3/projects/7a91e946f166455cad4ac3ce04da6517/users/f5b950be2fca4f538e7fd70804976b9b/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:48.964409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9f53a-130e-4887-ae66-324e596af65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a91e946f166455cad4ac3ce04da6517, user_id=f5b950be2fca4f538e7fd70804976b9b, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:48.967114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9f53a-130e-4887-ae66-324e596af65b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:48.981425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1128/2270] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:34:48 2022] PUT /identity/v3/projects/7a91e946f166455cad4ac3ce04da6517/users/f5b950be2fca4f538e7fd70804976b9b/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:48.985380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5074a-c572-4112-9f61-65d8e11b3faa admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:48.996096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5074a-c572-4112-9f61-65d8e11b3faa admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:49.009499 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:49.060203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48169c12-9293-4144-92d1-d0a5d6f26946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:49.060837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48169c12-9293-4144-92d1-d0a5d6f26946 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:49.061120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48169c12-9293-4144-92d1-d0a5d6f26946 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:49.061393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48169c12-9293-4144-92d1-d0a5d6f26946 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:49.062037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48169c12-9293-4144-92d1-d0a5d6f26946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:49.063639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48169c12-9293-4144-92d1-d0a5d6f26946 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:49.073747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1143/2271] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:34:49.078232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9f53a-130e-4887-ae66-324e596af65b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:49.088821 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9f53a-130e-4887-ae66-324e596af65b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:49.093314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a45f01-874e-4947-bc45-a030015c501a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:49.093949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a45f01-874e-4947-bc45-a030015c501a admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:49.094222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a45f01-874e-4947-bc45-a030015c501a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:49.094488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a45f01-874e-4947-bc45-a030015c501a admin admin] PATH_INFO: `/v3/projects/7a91e946f166455cad4ac3ce04da6517/users/f5b950be2fca4f538e7fd70804976b9b/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:49.096362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a45f01-874e-4947-bc45-a030015c501a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a91e946f166455cad4ac3ce04da6517, user_id=f5b950be2fca4f538e7fd70804976b9b, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:49.099811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a45f01-874e-4947-bc45-a030015c501a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:49.109762 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1129/2272] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:34:49 2022] PUT /identity/v3/projects/7a91e946f166455cad4ac3ce04da6517/users/f5b950be2fca4f538e7fd70804976b9b/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:49.121263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9199e7-ebe0-42da-8d83-608f814256f0 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:49.121624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9199e7-ebe0-42da-8d83-608f814256f0 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:49.121985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9199e7-ebe0-42da-8d83-608f814256f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:49.150350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bb9199e7-ebe0-42da-8d83-608f814256f0 None None] MFA Rules not processed for user `f5b950be2fca4f538e7fd70804976b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:49.151675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bb9199e7-ebe0-42da-8d83-608f814256f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:49.218721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1144/2273] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:49.230270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a45f01-874e-4947-bc45-a030015c501a admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:49.231958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-39a45f01-874e-4947-bc45-a030015c501a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:49.289026 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a45f01-874e-4947-bc45-a030015c501a admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:49.292682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249f12f-201c-4587-9fb6-38f2426ecae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:49.293283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249f12f-201c-4587-9fb6-38f2426ecae6 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:49.293559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249f12f-201c-4587-9fb6-38f2426ecae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:49.293828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249f12f-201c-4587-9fb6-38f2426ecae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:49.302648 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249f12f-201c-4587-9fb6-38f2426ecae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:49.304060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249f12f-201c-4587-9fb6-38f2426ecae6 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:49.313289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1130/2274] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:34:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:34:50.668740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9199e7-ebe0-42da-8d83-608f814256f0 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:50.730838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9199e7-ebe0-42da-8d83-608f814256f0 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:50.735263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:50.735919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:50.736193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:50.736464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] PATH_INFO: `/v3/users/e7e3cc07a8cc42e9bc2cbf71586d39bb` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:50.737591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e3cc07a8cc42e9bc2cbf71586d39bb)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:50.739813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:50.781852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:50.789432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e3cc07a8cc42e9bc2cbf71586d39bb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:50.790104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e3cc07a8cc42e9bc2cbf71586d39bb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:50.797754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e3cc07a8cc42e9bc2cbf71586d39bb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:50.805768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e3cc07a8cc42e9bc2cbf71586d39bb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:50.806441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e3cc07a8cc42e9bc2cbf71586d39bb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:50.825040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1145/2275] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:34:50 2022] DELETE /identity/v3/users/e7e3cc07a8cc42e9bc2cbf71586d39bb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:50.829763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec2fbc-eca5-4163-8c4d-ec25d255fb45 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:50.830281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec2fbc-eca5-4163-8c4d-ec25d255fb45 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:50.830469 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec2fbc-eca5-4163-8c4d-ec25d255fb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:50.850408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1bec2fbc-eca5-4163-8c4d-ec25d255fb45 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:50.851611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1bec2fbc-eca5-4163-8c4d-ec25d255fb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:50.910283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1131/2276] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:50.919836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:50.921366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:50.981058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6c2be-ffe8-439c-b614-f1bfa1b7630c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:50.984696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4993a-37cf-4bf7-ba6a-b9ab3c50dcf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:50.985298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4993a-37cf-4bf7-ba6a-b9ab3c50dcf9 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:50.985743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4993a-37cf-4bf7-ba6a-b9ab3c50dcf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:50.986020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4993a-37cf-4bf7-ba6a-b9ab3c50dcf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:50.995546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4993a-37cf-4bf7-ba6a-b9ab3c50dcf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:50.997008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4993a-37cf-4bf7-ba6a-b9ab3c50dcf9 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:51.006885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1146/2277] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:34:51.273405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec2fbc-eca5-4163-8c4d-ec25d255fb45 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:51.274973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1bec2fbc-eca5-4163-8c4d-ec25d255fb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:51.287543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec2fbc-eca5-4163-8c4d-ec25d255fb45 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:51.292873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:51.293773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:51.294112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:51.294561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] PATH_INFO: `/v3/projects/877691afa51e47bb8891a4de1c7c5cd2` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:51.296169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=877691afa51e47bb8891a4de1c7c5cd2)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:51.298480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:51.326443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:51.351439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '877691afa51e47bb8891a4de1c7c5cd2', 'request_id': 'req-d9aba319-2d97-40f8-aa14-96647e12c9f2'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.357084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '877691afa51e47bb8891a4de1c7c5cd2', 'request_id': 'req-d9aba319-2d97-40f8-aa14-96647e12c9f2'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.365418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '877691afa51e47bb8891a4de1c7c5cd2', 'request_id': 'req-d9aba319-2d97-40f8-aa14-96647e12c9f2'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.379855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] The token cache is being invalidate because project 877691afa51e47bb8891a4de1c7c5cd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:34:51.380144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.380897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.382480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1132/2278] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:34:51 2022] DELETE /identity/v3/projects/877691afa51e47bb8891a4de1c7c5cd2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:51.386566 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4993a-37cf-4bf7-ba6a-b9ab3c50dcf9 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:51.396628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4993a-37cf-4bf7-ba6a-b9ab3c50dcf9 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:51.454171 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:51.454807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:51.455086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:51.455356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] PATH_INFO: `/v3/users/f5b950be2fca4f538e7fd70804976b9b` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:51.456471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b950be2fca4f538e7fd70804976b9b)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:51.459155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:51.506457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b950be2fca4f538e7fd70804976b9b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.507106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b950be2fca4f538e7fd70804976b9b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.514134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b950be2fca4f538e7fd70804976b9b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.521895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b950be2fca4f538e7fd70804976b9b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.522556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0704a2fd-d85f-4bad-a34b-b62054f191e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b950be2fca4f538e7fd70804976b9b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.523680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:51.542466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1147/2279] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:34:51 2022] DELETE /identity/v3/users/f5b950be2fca4f538e7fd70804976b9b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:51.819874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:51.821424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:51.835336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aba319-2d97-40f8-aa14-96647e12c9f2 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:51.892997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:51.893562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:51.893837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:51.894103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] PATH_INFO: `/v3/projects/7a91e946f166455cad4ac3ce04da6517` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:51.896218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a91e946f166455cad4ac3ce04da6517)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:51.898197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:51.944539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a91e946f166455cad4ac3ce04da6517', 'request_id': 'req-8811283a-4d3c-492e-bfb0-080521f80584'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.950106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a91e946f166455cad4ac3ce04da6517', 'request_id': 'req-8811283a-4d3c-492e-bfb0-080521f80584'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.957323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a91e946f166455cad4ac3ce04da6517', 'request_id': 'req-8811283a-4d3c-492e-bfb0-080521f80584'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.970320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] The token cache is being invalidate because project 7a91e946f166455cad4ac3ce04da6517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:34:51.970619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.971231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8811283a-4d3c-492e-bfb0-080521f80584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:34:51.973474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1133/2280] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:34:51 2022] DELETE /identity/v3/projects/7a91e946f166455cad4ac3ce04da6517 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:51.978468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad57446-9bea-495c-adbb-306c03c89566 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:51.978927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad57446-9bea-495c-adbb-306c03c89566 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:51.979133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad57446-9bea-495c-adbb-306c03c89566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:51.998413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5ad57446-9bea-495c-adbb-306c03c89566 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:51.999632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5ad57446-9bea-495c-adbb-306c03c89566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:52.061787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1148/2281] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:52.069764 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db229b-d9ce-4336-a00e-9c2e62be5620 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.070048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db229b-d9ce-4336-a00e-9c2e62be5620 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.070320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db229b-d9ce-4336-a00e-9c2e62be5620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.090348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-63db229b-d9ce-4336-a00e-9c2e62be5620 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:52.091537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-63db229b-d9ce-4336-a00e-9c2e62be5620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:52.099975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1134/2282] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:52.104259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad57446-9bea-495c-adbb-306c03c89566 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.114810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad57446-9bea-495c-adbb-306c03c89566 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.119441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8d8f1-9159-4362-b7a6-d31d3e5c6c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.120122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8d8f1-9159-4362-b7a6-d31d3e5c6c64 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.120299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8d8f1-9159-4362-b7a6-d31d3e5c6c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.120668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8d8f1-9159-4362-b7a6-d31d3e5c6c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.121324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8d8f1-9159-4362-b7a6-d31d3e5c6c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:34:52.121695 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8d8f1-9159-4362-b7a6-d31d3e5c6c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.123303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8d8f1-9159-4362-b7a6-d31d3e5c6c64 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.135813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1149/2283] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:34:52.140400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db229b-d9ce-4336-a00e-9c2e62be5620 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.149461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db229b-d9ce-4336-a00e-9c2e62be5620 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.153905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a243f41-154c-41f0-a464-dc4369f24714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.154486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a243f41-154c-41f0-a464-dc4369f24714 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.154759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a243f41-154c-41f0-a464-dc4369f24714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.155021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a243f41-154c-41f0-a464-dc4369f24714 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.155629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a243f41-154c-41f0-a464-dc4369f24714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.157561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a243f41-154c-41f0-a464-dc4369f24714 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.184625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1135/2284] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:34:52 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:34:52.188527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8d8f1-9159-4362-b7a6-d31d3e5c6c64 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.199109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8d8f1-9159-4362-b7a6-d31d3e5c6c64 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.257747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4a45f-460a-4863-93b7-2e5e6a18b95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.258360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a45f-460a-4863-93b7-2e5e6a18b95d admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.258636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a45f-460a-4863-93b7-2e5e6a18b95d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.258906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4a45f-460a-4863-93b7-2e5e6a18b95d admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.259538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4a45f-460a-4863-93b7-2e5e6a18b95d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.261492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4a45f-460a-4863-93b7-2e5e6a18b95d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.282974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1150/2285] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:34:52 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:34:52.287119 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a243f41-154c-41f0-a464-dc4369f24714 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.297426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a243f41-154c-41f0-a464-dc4369f24714 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.302812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b3819-8136-4829-905b-1815abc45422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.303403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b3819-8136-4829-905b-1815abc45422 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.303674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b3819-8136-4829-905b-1815abc45422 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.303939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b3819-8136-4829-905b-1815abc45422 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.304605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b3819-8136-4829-905b-1815abc45422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.306186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b3819-8136-4829-905b-1815abc45422 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.316992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1136/2286] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:34:52.320859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4a45f-460a-4863-93b7-2e5e6a18b95d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.331069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4a45f-460a-4863-93b7-2e5e6a18b95d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.335685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3db2b5-f9c3-4a67-bcaf-d91d1efa130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.336279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3db2b5-f9c3-4a67-bcaf-d91d1efa130b admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.336592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3db2b5-f9c3-4a67-bcaf-d91d1efa130b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.336868 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3db2b5-f9c3-4a67-bcaf-d91d1efa130b admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.337549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3db2b5-f9c3-4a67-bcaf-d91d1efa130b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.339287 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3db2b5-f9c3-4a67-bcaf-d91d1efa130b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.350949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1151/2287] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:34:52.355337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b3819-8136-4829-905b-1815abc45422 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.360360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:52.366420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b3819-8136-4829-905b-1815abc45422 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.372170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211fb9f-bb59-4723-a757-f53c5e4c95d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.372853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211fb9f-bb59-4723-a757-f53c5e4c95d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.373133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211fb9f-bb59-4723-a757-f53c5e4c95d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.373406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211fb9f-bb59-4723-a757-f53c5e4c95d3 admin admin] PATH_INFO: `/v3/projects/a1b0bb112cb54d1d940670e38d181eb7/users/1f67d1e48e224c44b88a74c38075071c/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.388461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211fb9f-bb59-4723-a757-f53c5e4c95d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b0bb112cb54d1d940670e38d181eb7, user_id=1f67d1e48e224c44b88a74c38075071c, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.391154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211fb9f-bb59-4723-a757-f53c5e4c95d3 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.405083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1137/2288] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:34:52 2022] PUT /identity/v3/projects/a1b0bb112cb54d1d940670e38d181eb7/users/1f67d1e48e224c44b88a74c38075071c/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:52.416342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0058298-6359-431a-8cdb-7abc98607098 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.416669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0058298-6359-431a-8cdb-7abc98607098 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.416949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0058298-6359-431a-8cdb-7abc98607098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.445955 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a0058298-6359-431a-8cdb-7abc98607098 None None] MFA Rules not processed for user `1f67d1e48e224c44b88a74c38075071c`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:52.447262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a0058298-6359-431a-8cdb-7abc98607098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:52.481764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:52.520297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1152/2289] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:52.524437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211fb9f-bb59-4723-a757-f53c5e4c95d3 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.596804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211fb9f-bb59-4723-a757-f53c5e4c95d3 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.602944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97518c4a-c525-457e-90ea-5460126fc007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.603558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97518c4a-c525-457e-90ea-5460126fc007 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.603836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97518c4a-c525-457e-90ea-5460126fc007 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.604105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97518c4a-c525-457e-90ea-5460126fc007 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.604775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97518c4a-c525-457e-90ea-5460126fc007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.606678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97518c4a-c525-457e-90ea-5460126fc007 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.635134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1138/2290] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:34:52 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:34:52.639215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0058298-6359-431a-8cdb-7abc98607098 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.650187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0058298-6359-431a-8cdb-7abc98607098 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.708822 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b4949-6837-40bf-8511-8f4a046baaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.709356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b4949-6837-40bf-8511-8f4a046baaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.709635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b4949-6837-40bf-8511-8f4a046baaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.709909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b4949-6837-40bf-8511-8f4a046baaa6 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.710548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b4949-6837-40bf-8511-8f4a046baaa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.712540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b4949-6837-40bf-8511-8f4a046baaa6 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.735063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1153/2291] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:34:52 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:34:52.739937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97518c4a-c525-457e-90ea-5460126fc007 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.750478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97518c4a-c525-457e-90ea-5460126fc007 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.755314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530bd05-6628-40c7-b629-31abea0cc97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.755918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530bd05-6628-40c7-b629-31abea0cc97f admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.756196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530bd05-6628-40c7-b629-31abea0cc97f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.756466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530bd05-6628-40c7-b629-31abea0cc97f admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.757198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530bd05-6628-40c7-b629-31abea0cc97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.758792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530bd05-6628-40c7-b629-31abea0cc97f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.770001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1139/2292] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:34:52.774681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b4949-6837-40bf-8511-8f4a046baaa6 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.784595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b4949-6837-40bf-8511-8f4a046baaa6 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.790133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60bfe5-d64c-40bc-b430-003764062c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.790775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60bfe5-d64c-40bc-b430-003764062c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.791058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60bfe5-d64c-40bc-b430-003764062c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.791332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60bfe5-d64c-40bc-b430-003764062c8e admin admin] PATH_INFO: `/v3/projects/b10264182cfa4019a5e0c5531638be8e/users/8b2017532a0944fa9ab6e0e14f4af441/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.806334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60bfe5-d64c-40bc-b430-003764062c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10264182cfa4019a5e0c5531638be8e, user_id=8b2017532a0944fa9ab6e0e14f4af441, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.809069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60bfe5-d64c-40bc-b430-003764062c8e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.822281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1154/2293] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:34:52 2022] PUT /identity/v3/projects/b10264182cfa4019a5e0c5531638be8e/users/8b2017532a0944fa9ab6e0e14f4af441/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:52.826380 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530bd05-6628-40c7-b629-31abea0cc97f admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.837030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530bd05-6628-40c7-b629-31abea0cc97f admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.891559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1263fe-cb68-4e47-a638-9e9a701f72d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.892162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1263fe-cb68-4e47-a638-9e9a701f72d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.892433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1263fe-cb68-4e47-a638-9e9a701f72d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.892741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1263fe-cb68-4e47-a638-9e9a701f72d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.893372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1263fe-cb68-4e47-a638-9e9a701f72d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.895692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1263fe-cb68-4e47-a638-9e9a701f72d9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.905461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1140/2294] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:34:52.911701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60bfe5-d64c-40bc-b430-003764062c8e admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:52.922310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60bfe5-d64c-40bc-b430-003764062c8e admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:52.927021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8ea4c-14b1-42c3-ab5e-95315af957dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:52.927660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8ea4c-14b1-42c3-ab5e-95315af957dd admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.927937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8ea4c-14b1-42c3-ab5e-95315af957dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.928209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8ea4c-14b1-42c3-ab5e-95315af957dd admin admin] PATH_INFO: `/v3/projects/b10264182cfa4019a5e0c5531638be8e/users/8b2017532a0944fa9ab6e0e14f4af441/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.930140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8ea4c-14b1-42c3-ab5e-95315af957dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10264182cfa4019a5e0c5531638be8e, user_id=8b2017532a0944fa9ab6e0e14f4af441, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:52.932827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8ea4c-14b1-42c3-ab5e-95315af957dd admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:52.942833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1155/2295] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:34:52 2022] PUT /identity/v3/projects/b10264182cfa4019a5e0c5531638be8e/users/8b2017532a0944fa9ab6e0e14f4af441/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:34:52.953308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927015d7-0626-43c3-891b-419608661c8e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:52.953585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927015d7-0626-43c3-891b-419608661c8e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:52.953875 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927015d7-0626-43c3-891b-419608661c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:52.981888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-927015d7-0626-43c3-891b-419608661c8e None None] MFA Rules not processed for user `8b2017532a0944fa9ab6e0e14f4af441`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:34:52.982997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-927015d7-0626-43c3-891b-419608661c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:53.019183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:53.050282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1141/2296] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:34:53.061272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8ea4c-14b1-42c3-ab5e-95315af957dd admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:53.063045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-01c8ea4c-14b1-42c3-ab5e-95315af957dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:53.121741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8ea4c-14b1-42c3-ab5e-95315af957dd admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:53.125331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463ee27-668e-4d1f-bb03-7c62aa3edae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:53.125917 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463ee27-668e-4d1f-bb03-7c62aa3edae7 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:53.126189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463ee27-668e-4d1f-bb03-7c62aa3edae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:53.126456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463ee27-668e-4d1f-bb03-7c62aa3edae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:53.170424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:34:53.186360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463ee27-668e-4d1f-bb03-7c62aa3edae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:53.187812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463ee27-668e-4d1f-bb03-7c62aa3edae7 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:53.197771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1156/2297] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:34:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:34:58.838596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927015d7-0626-43c3-891b-419608661c8e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:34:58.839976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-927015d7-0626-43c3-891b-419608661c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:34:58.852458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927015d7-0626-43c3-891b-419608661c8e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:34:58.855999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fccbf-0201-4bc5-8ac4-7e0d5170b395 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:34:58.856629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fccbf-0201-4bc5-8ac4-7e0d5170b395 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:34:58.856913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fccbf-0201-4bc5-8ac4-7e0d5170b395 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:34:58.857177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fccbf-0201-4bc5-8ac4-7e0d5170b395 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:34:58.866251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fccbf-0201-4bc5-8ac4-7e0d5170b395 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:34:58.867663 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fccbf-0201-4bc5-8ac4-7e0d5170b395 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:34:58.877011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1142/2298] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:34:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:03.193539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=8 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:04.275219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463ee27-668e-4d1f-bb03-7c62aa3edae7 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:04.276734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0463ee27-668e-4d1f-bb03-7c62aa3edae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:04.349637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463ee27-668e-4d1f-bb03-7c62aa3edae7 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:04.355120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413b9b1-674a-4367-9117-3ae770e8c19d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:04.355703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413b9b1-674a-4367-9117-3ae770e8c19d service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:04.355981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413b9b1-674a-4367-9117-3ae770e8c19d service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:04.356253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413b9b1-674a-4367-9117-3ae770e8c19d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:04.358001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4413b9b1-674a-4367-9117-3ae770e8c19d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:04.422031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413b9b1-674a-4367-9117-3ae770e8c19d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:04.423652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413b9b1-674a-4367-9117-3ae770e8c19d service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:04.432621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1157/2299] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:06.225376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fccbf-0201-4bc5-8ac4-7e0d5170b395 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:06.226878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ec5fccbf-0201-4bc5-8ac4-7e0d5170b395 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:06.242609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fccbf-0201-4bc5-8ac4-7e0d5170b395 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:06.248030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:06.248681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:06.248962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:06.249251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:06.250915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:06.308832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:06.310416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:06.319561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1143/2300] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:14.428569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=56 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:16.315117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=46 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:19.558179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865dfc7-aec1-48ca-85ac-c20679022f3f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:19.558704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865dfc7-aec1-48ca-85ac-c20679022f3f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:19.558704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865dfc7-aec1-48ca-85ac-c20679022f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:19.579162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b865dfc7-aec1-48ca-85ac-c20679022f3f None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:19.580383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b865dfc7-aec1-48ca-85ac-c20679022f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:19.636526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1158/2301] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:35:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:19.684520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:19.685905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:19.763931 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d66b08-ebc5-4898-a85a-564eb4cc760a service nova] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:19.770537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9f77-7cfe-4509-af5c-20c84cd0ea25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:19.771296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9f77-7cfe-4509-af5c-20c84cd0ea25 service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:19.771561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9f77-7cfe-4509-af5c-20c84cd0ea25 service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:19.771828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab9f77-7cfe-4509-af5c-20c84cd0ea25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:19.780619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab9f77-7cfe-4509-af5c-20c84cd0ea25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:19.782213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab9f77-7cfe-4509-af5c-20c84cd0ea25 service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:19.790991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1144/2302] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:35:23.654126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865dfc7-aec1-48ca-85ac-c20679022f3f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:23.676989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865dfc7-aec1-48ca-85ac-c20679022f3f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:23.706196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1e29d-6a5c-4751-b068-1bb9eed46cad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:23.707932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1e29d-6a5c-4751-b068-1bb9eed46cad service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:23.708891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1e29d-6a5c-4751-b068-1bb9eed46cad service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:23.709354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1e29d-6a5c-4751-b068-1bb9eed46cad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:23.711437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-20e1e29d-6a5c-4751-b068-1bb9eed46cad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:23.736576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1e29d-6a5c-4751-b068-1bb9eed46cad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:23.739480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1e29d-6a5c-4751-b068-1bb9eed46cad service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:23.758984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1159/2303] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:24.295887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9f77-7cfe-4509-af5c-20c84cd0ea25 service swift] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:24.415290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab9f77-7cfe-4509-af5c-20c84cd0ea25 service swift] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:24.420999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:24.422870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:24.423514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:24.424237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] PATH_INFO: `/v3/users/1f67d1e48e224c44b88a74c38075071c` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:24.426959 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f67d1e48e224c44b88a74c38075071c)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:24.430188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:24.522017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f67d1e48e224c44b88a74c38075071c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:24.523375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f67d1e48e224c44b88a74c38075071c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:24.541855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f67d1e48e224c44b88a74c38075071c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:24.550493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f67d1e48e224c44b88a74c38075071c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:24.560527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f67d1e48e224c44b88a74c38075071c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:24.572794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1145/2304] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:35:24 2022] DELETE /identity/v3/users/1f67d1e48e224c44b88a74c38075071c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:24.576659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bc548-b412-46a1-93d5-27e13b8d7dd2 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:24.576950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bc548-b412-46a1-93d5-27e13b8d7dd2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:24.577305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bc548-b412-46a1-93d5-27e13b8d7dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:24.605319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-906bc548-b412-46a1-93d5-27e13b8d7dd2 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:24.606969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-906bc548-b412-46a1-93d5-27e13b8d7dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:24.685620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1160/2305] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:24.696621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:24.698136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:24.710768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:24.792583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdbbde-e4b5-489f-9065-534a2e43cf22 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:24.796296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54553ed-3e53-46b1-a446-629c4f4e7ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:24.796952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54553ed-3e53-46b1-a446-629c4f4e7ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:24.797377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54553ed-3e53-46b1-a446-629c4f4e7ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:24.797650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54553ed-3e53-46b1-a446-629c4f4e7ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:24.808094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54553ed-3e53-46b1-a446-629c4f4e7ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:24.809591 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54553ed-3e53-46b1-a446-629c4f4e7ea4 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:24.821327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1146/2306] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:25.177610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bc548-b412-46a1-93d5-27e13b8d7dd2 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:25.179526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-906bc548-b412-46a1-93d5-27e13b8d7dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:25.213707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bc548-b412-46a1-93d5-27e13b8d7dd2 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:25.235534 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:25.236270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:25.236652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:25.236999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] PATH_INFO: `/v3/projects/a1b0bb112cb54d1d940670e38d181eb7` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:25.238535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b0bb112cb54d1d940670e38d181eb7)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:25.240589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:25.315119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b0bb112cb54d1d940670e38d181eb7', 'request_id': 'req-d2d4dd25-8250-4b13-9e07-d2969232ec8f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.323437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b0bb112cb54d1d940670e38d181eb7', 'request_id': 'req-d2d4dd25-8250-4b13-9e07-d2969232ec8f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.333959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b0bb112cb54d1d940670e38d181eb7', 'request_id': 'req-d2d4dd25-8250-4b13-9e07-d2969232ec8f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.364047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] The token cache is being invalidate because project a1b0bb112cb54d1d940670e38d181eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:35:25.364711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.366724 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.369337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1161/2307] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:35:25 2022] DELETE /identity/v3/projects/a1b0bb112cb54d1d940670e38d181eb7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:25.373502 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54553ed-3e53-46b1-a446-629c4f4e7ea4 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:25.399284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54553ed-3e53-46b1-a446-629c4f4e7ea4 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:25.498137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:25.498908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:25.499238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:25.499557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] PATH_INFO: `/v3/users/8b2017532a0944fa9ab6e0e14f4af441` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:25.500977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2017532a0944fa9ab6e0e14f4af441)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:25.502992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:25.578706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2017532a0944fa9ab6e0e14f4af441'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.579347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2017532a0944fa9ab6e0e14f4af441'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.596619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2017532a0944fa9ab6e0e14f4af441'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.608298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2017532a0944fa9ab6e0e14f4af441'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.621363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-aad9878c-1386-4b59-bc94-c386571373e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2017532a0944fa9ab6e0e14f4af441'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:25.630768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1147/2308] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:35:25 2022] DELETE /identity/v3/users/8b2017532a0944fa9ab6e0e14f4af441 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:26.116615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:26.118747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:26.134739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4dd25-8250-4b13-9e07-d2969232ec8f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:26.160689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:26.245788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:26.247115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:26.247823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:26.248946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] PATH_INFO: `/v3/projects/b10264182cfa4019a5e0c5531638be8e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:26.257497 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10264182cfa4019a5e0c5531638be8e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:26.259882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:26.345181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10264182cfa4019a5e0c5531638be8e', 'request_id': 'req-733d924e-67e0-4c09-b398-d27b7b0cba5f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:26.356218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10264182cfa4019a5e0c5531638be8e', 'request_id': 'req-733d924e-67e0-4c09-b398-d27b7b0cba5f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:26.365473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10264182cfa4019a5e0c5531638be8e', 'request_id': 'req-733d924e-67e0-4c09-b398-d27b7b0cba5f'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:26.383728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] The token cache is being invalidate because project b10264182cfa4019a5e0c5531638be8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:35:26.384038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:26.384760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-733d924e-67e0-4c09-b398-d27b7b0cba5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:26.388702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1162/2309] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:35:26 2022] DELETE /identity/v3/projects/b10264182cfa4019a5e0c5531638be8e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:26.392022 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38cf45-0b8e-4f5d-8987-72c23b5cd365 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:26.392842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38cf45-0b8e-4f5d-8987-72c23b5cd365 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:26.393114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38cf45-0b8e-4f5d-8987-72c23b5cd365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:26.415464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2b38cf45-0b8e-4f5d-8987-72c23b5cd365 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:26.416958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2b38cf45-0b8e-4f5d-8987-72c23b5cd365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:26.447742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:26.511326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1148/2310] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:26.513250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930b098-31c6-43b1-9cf7-9e14c2ba049a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:26.513520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930b098-31c6-43b1-9cf7-9e14c2ba049a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:26.513805 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930b098-31c6-43b1-9cf7-9e14c2ba049a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:26.542582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-9930b098-31c6-43b1-9cf7-9e14c2ba049a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:26.543852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9930b098-31c6-43b1-9cf7-9e14c2ba049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:26.565939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1163/2311] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:35:26.570468 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38cf45-0b8e-4f5d-8987-72c23b5cd365 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:26.581635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38cf45-0b8e-4f5d-8987-72c23b5cd365 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:26.588028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51584a4-5229-4cb4-8303-532c81127dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:26.588908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51584a4-5229-4cb4-8303-532c81127dff admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:26.589371 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51584a4-5229-4cb4-8303-532c81127dff admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:26.589960 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51584a4-5229-4cb4-8303-532c81127dff admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:26.590915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51584a4-5229-4cb4-8303-532c81127dff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:35:26.591227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51584a4-5229-4cb4-8303-532c81127dff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:26.592933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51584a4-5229-4cb4-8303-532c81127dff admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:26.611151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930b098-31c6-43b1-9cf7-9e14c2ba049a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:26.617539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1149/2312] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:35:26.622975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930b098-31c6-43b1-9cf7-9e14c2ba049a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:26.629357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fbe08-ee14-49d0-bebb-cff8a6ea9138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:26.630066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fbe08-ee14-49d0-bebb-cff8a6ea9138 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:26.630550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fbe08-ee14-49d0-bebb-cff8a6ea9138 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:26.630886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fbe08-ee14-49d0-bebb-cff8a6ea9138 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:26.633722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fbe08-ee14-49d0-bebb-cff8a6ea9138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:26.638699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fbe08-ee14-49d0-bebb-cff8a6ea9138 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:26.696857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1164/2313] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:35:26 2022] POST /identity/v3/projects => generated 391 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:26.701281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51584a4-5229-4cb4-8303-532c81127dff admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:26.719635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51584a4-5229-4cb4-8303-532c81127dff admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:26.814564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffac2b-2df9-40ef-9e5b-1a4155613d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:26.815365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffac2b-2df9-40ef-9e5b-1a4155613d1e admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:26.815743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffac2b-2df9-40ef-9e5b-1a4155613d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:26.816174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffac2b-2df9-40ef-9e5b-1a4155613d1e admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:26.817008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffac2b-2df9-40ef-9e5b-1a4155613d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:26.819371 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffac2b-2df9-40ef-9e5b-1a4155613d1e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:26.854964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1150/2314] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:35:26 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:26.860092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fbe08-ee14-49d0-bebb-cff8a6ea9138 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:26.874543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fbe08-ee14-49d0-bebb-cff8a6ea9138 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:26.881650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8ccfb-fe82-46e4-a9a3-af653c82b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:26.882376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8ccfb-fe82-46e4-a9a3-af653c82b39c admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:26.882666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8ccfb-fe82-46e4-a9a3-af653c82b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:26.883027 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8ccfb-fe82-46e4-a9a3-af653c82b39c admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:26.883730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8ccfb-fe82-46e4-a9a3-af653c82b39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:26.896573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8ccfb-fe82-46e4-a9a3-af653c82b39c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:26.901208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1165/2315] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:26.906112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffac2b-2df9-40ef-9e5b-1a4155613d1e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:26.917020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffac2b-2df9-40ef-9e5b-1a4155613d1e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:26.925130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd3df0-f565-444d-b4d6-169cca07bf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:26.933133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd3df0-f565-444d-b4d6-169cca07bf6d admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:26.933133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd3df0-f565-444d-b4d6-169cca07bf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:26.933133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd3df0-f565-444d-b4d6-169cca07bf6d admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:26.933133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd3df0-f565-444d-b4d6-169cca07bf6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:26.933133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd3df0-f565-444d-b4d6-169cca07bf6d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:26.949290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8ccfb-fe82-46e4-a9a3-af653c82b39c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:26.950423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1151/2316] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:26.970771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8ccfb-fe82-46e4-a9a3-af653c82b39c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:26.976488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d8b11-1944-4cd0-a2c2-3d75c929f0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:26.977710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d8b11-1944-4cd0-a2c2-3d75c929f0fa admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:26.977710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d8b11-1944-4cd0-a2c2-3d75c929f0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:26.977710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d8b11-1944-4cd0-a2c2-3d75c929f0fa admin admin] PATH_INFO: `/v3/projects/197f8c3a5eb6418c8e0c66d43f6fd958/users/553ec3d9204b4e9baa92839c15587e93/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:26.995057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d8b11-1944-4cd0-a2c2-3d75c929f0fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=197f8c3a5eb6418c8e0c66d43f6fd958, user_id=553ec3d9204b4e9baa92839c15587e93, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:26.999778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d8b11-1944-4cd0-a2c2-3d75c929f0fa admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:27.020699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1166/2317] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:35:26 2022] PUT /identity/v3/projects/197f8c3a5eb6418c8e0c66d43f6fd958/users/553ec3d9204b4e9baa92839c15587e93/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:27.026675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff97e-e3fd-4e55-96b6-631e92c73494 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:27.026961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff97e-e3fd-4e55-96b6-631e92c73494 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:27.027238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bff97e-e3fd-4e55-96b6-631e92c73494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:27.063106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-77bff97e-e3fd-4e55-96b6-631e92c73494 None None] MFA Rules not processed for user `553ec3d9204b4e9baa92839c15587e93`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:27.064696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-77bff97e-e3fd-4e55-96b6-631e92c73494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:27.164394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1152/2318] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:27.168712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d8b11-1944-4cd0-a2c2-3d75c929f0fa admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:27.223823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:27.266051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d8b11-1944-4cd0-a2c2-3d75c929f0fa admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:27.287178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01734284-c8b9-4587-9434-305495f88ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:27.287178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01734284-c8b9-4587-9434-305495f88ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:27.287178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01734284-c8b9-4587-9434-305495f88ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:27.287178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01734284-c8b9-4587-9434-305495f88ed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:27.287693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01734284-c8b9-4587-9434-305495f88ed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:27.288315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01734284-c8b9-4587-9434-305495f88ed2 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:27.323984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1167/2319] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:35:27 2022] POST /identity/v3/projects => generated 389 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:27.329015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bff97e-e3fd-4e55-96b6-631e92c73494 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:27.346229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bff97e-e3fd-4e55-96b6-631e92c73494 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:27.392118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:27.438095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f6748-928b-4d18-a769-6403f849c3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:27.438939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f6748-928b-4d18-a769-6403f849c3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:27.439363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f6748-928b-4d18-a769-6403f849c3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:27.439597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f6748-928b-4d18-a769-6403f849c3d4 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:27.440795 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f6748-928b-4d18-a769-6403f849c3d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:27.442546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f6748-928b-4d18-a769-6403f849c3d4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:27.471549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1153/2320] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:35:27 2022] POST /identity/v3/users => generated 301 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:35:27.476565 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01734284-c8b9-4587-9434-305495f88ed2 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:27.488286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01734284-c8b9-4587-9434-305495f88ed2 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:27.493790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470ad75-1ada-45a2-8842-cf91f2295986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:27.494791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470ad75-1ada-45a2-8842-cf91f2295986 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:27.494791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470ad75-1ada-45a2-8842-cf91f2295986 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:27.495016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470ad75-1ada-45a2-8842-cf91f2295986 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:27.495843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470ad75-1ada-45a2-8842-cf91f2295986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:27.497568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470ad75-1ada-45a2-8842-cf91f2295986 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:27.511151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1168/2321] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:27.516148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f6748-928b-4d18-a769-6403f849c3d4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:27.525684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f6748-928b-4d18-a769-6403f849c3d4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:27.531222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290bcb76-750a-4f27-a6b1-a70abac43087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:27.531808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bcb76-750a-4f27-a6b1-a70abac43087 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:27.532212 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bcb76-750a-4f27-a6b1-a70abac43087 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:27.532630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bcb76-750a-4f27-a6b1-a70abac43087 admin admin] PATH_INFO: `/v3/projects/275b335f84e6499a84e149821492cc1f/users/91019cd9e99d4a4b856826e190bec0cb/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:27.549964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290bcb76-750a-4f27-a6b1-a70abac43087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275b335f84e6499a84e149821492cc1f, user_id=91019cd9e99d4a4b856826e190bec0cb, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:27.553859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290bcb76-750a-4f27-a6b1-a70abac43087 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:27.573035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1154/2322] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:35:27 2022] PUT /identity/v3/projects/275b335f84e6499a84e149821492cc1f/users/91019cd9e99d4a4b856826e190bec0cb/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:27.577209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470ad75-1ada-45a2-8842-cf91f2295986 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:27.593210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470ad75-1ada-45a2-8842-cf91f2295986 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:27.674479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512623e-f593-464a-a503-4a8f52aab11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:27.675069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512623e-f593-464a-a503-4a8f52aab11f admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:27.675370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512623e-f593-464a-a503-4a8f52aab11f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:27.675690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512623e-f593-464a-a503-4a8f52aab11f admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:27.677052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512623e-f593-464a-a503-4a8f52aab11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:27.678760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512623e-f593-464a-a503-4a8f52aab11f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:27.690150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1169/2323] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:27.694175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290bcb76-750a-4f27-a6b1-a70abac43087 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:27.706011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290bcb76-750a-4f27-a6b1-a70abac43087 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:27.711580 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259005c-321a-440b-80fb-196b558d31c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:27.712238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259005c-321a-440b-80fb-196b558d31c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:27.713198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259005c-321a-440b-80fb-196b558d31c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:27.713470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259005c-321a-440b-80fb-196b558d31c8 admin admin] PATH_INFO: `/v3/projects/275b335f84e6499a84e149821492cc1f/users/91019cd9e99d4a4b856826e190bec0cb/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:27.716652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259005c-321a-440b-80fb-196b558d31c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275b335f84e6499a84e149821492cc1f, user_id=91019cd9e99d4a4b856826e190bec0cb, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:27.719692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259005c-321a-440b-80fb-196b558d31c8 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:27.733950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1155/2324] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:35:27 2022] PUT /identity/v3/projects/275b335f84e6499a84e149821492cc1f/users/91019cd9e99d4a4b856826e190bec0cb/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:27.745623 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75014c-c4ab-41c4-916e-1e88fd3c3ced None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:27.745929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75014c-c4ab-41c4-916e-1e88fd3c3ced None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:27.746177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75014c-c4ab-41c4-916e-1e88fd3c3ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:27.825479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-4d75014c-c4ab-41c4-916e-1e88fd3c3ced None None] MFA Rules not processed for user `91019cd9e99d4a4b856826e190bec0cb`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:27.828046 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4d75014c-c4ab-41c4-916e-1e88fd3c3ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:27.929946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1170/2325] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:27 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:27.939085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259005c-321a-440b-80fb-196b558d31c8 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:27.940898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8259005c-321a-440b-80fb-196b558d31c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:28.046639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259005c-321a-440b-80fb-196b558d31c8 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:28.050972 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa19682-6ea3-4205-aa69-f323e0a1d683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:28.051711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa19682-6ea3-4205-aa69-f323e0a1d683 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:28.052000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa19682-6ea3-4205-aa69-f323e0a1d683 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:28.052414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa19682-6ea3-4205-aa69-f323e0a1d683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:28.063492 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa19682-6ea3-4205-aa69-f323e0a1d683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:28.065272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa19682-6ea3-4205-aa69-f323e0a1d683 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:28.088590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1156/2326] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:27 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:28.139305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75014c-c4ab-41c4-916e-1e88fd3c3ced None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:28.140831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4d75014c-c4ab-41c4-916e-1e88fd3c3ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:28.165665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75014c-c4ab-41c4-916e-1e88fd3c3ced None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:28.184012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1326b1-0156-499f-86ac-4e0d91f2c122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:28.185693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1326b1-0156-499f-86ac-4e0d91f2c122 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:28.186185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1326b1-0156-499f-86ac-4e0d91f2c122 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:28.186677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1326b1-0156-499f-86ac-4e0d91f2c122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:28.243517 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:28.289414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1326b1-0156-499f-86ac-4e0d91f2c122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:28.291173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1326b1-0156-499f-86ac-4e0d91f2c122 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:28.312004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1171/2327] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:28 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:28.566741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763a35e-fd52-496e-beed-35e48ec5942e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:28.567330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763a35e-fd52-496e-beed-35e48ec5942e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:28.567432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763a35e-fd52-496e-beed-35e48ec5942e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:28.590710 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3763a35e-fd52-496e-beed-35e48ec5942e None None] MFA Rules not processed for user `e7a36e4630904f4eac08c9de1b8235ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:28.592183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3763a35e-fd52-496e-beed-35e48ec5942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:28.682567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1157/2328] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:29.122158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1326b1-0156-499f-86ac-4e0d91f2c122 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:29.124331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2c1326b1-0156-499f-86ac-4e0d91f2c122 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:29.199551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1326b1-0156-499f-86ac-4e0d91f2c122 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:29.204204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683783a-d838-41fa-9887-45e2c1b99b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:29.204898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683783a-d838-41fa-9887-45e2c1b99b68 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:29.205176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683783a-d838-41fa-9887-45e2c1b99b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:29.205446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683783a-d838-41fa-9887-45e2c1b99b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:29.207412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9683783a-d838-41fa-9887-45e2c1b99b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:29.218671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683783a-d838-41fa-9887-45e2c1b99b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:29.220072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683783a-d838-41fa-9887-45e2c1b99b68 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:29.237452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1172/2329] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:29.711617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a8369-c46c-4161-bd6d-ca7268ffc29b None None] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:29.712648 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a8369-c46c-4161-bd6d-ca7268ffc29b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:29.713078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a8369-c46c-4161-bd6d-ca7268ffc29b None None] PATH_INFO: `/` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:29.714915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1158/2330] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 1 00:35:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:35:29.730328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0e29f-2eff-4a02-9903-7fc6def58cfa None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:29.730615 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0e29f-2eff-4a02-9903-7fc6def58cfa None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:29.731647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0e29f-2eff-4a02-9903-7fc6def58cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:29.770314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b8a0e29f-2eff-4a02-9903-7fc6def58cfa None None] MFA Rules not processed for user `e30caaea6d104a53a8654442c2383b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:29.771864 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b8a0e29f-2eff-4a02-9903-7fc6def58cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:29.860527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1173/2331] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 1 00:35:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:35:29.874366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a8369-c46c-4161-bd6d-ca7268ffc29b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:29.931412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a8369-c46c-4161-bd6d-ca7268ffc29b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:29.953844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf647d-77bc-4443-8a31-87bc64d2f791 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:29.957069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf647d-77bc-4443-8a31-87bc64d2f791 service placement] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:29.957352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf647d-77bc-4443-8a31-87bc64d2f791 service placement] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:29.958547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf647d-77bc-4443-8a31-87bc64d2f791 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:29.962567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1faf647d-77bc-4443-8a31-87bc64d2f791 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:29.990444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf647d-77bc-4443-8a31-87bc64d2f791 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:29.992484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf647d-77bc-4443-8a31-87bc64d2f791 service placement] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:30.063513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1159/2332] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:32.437076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0e29f-2eff-4a02-9903-7fc6def58cfa None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:32.451353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf647d-77bc-4443-8a31-87bc64d2f791 service placement] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:32.463023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf647d-77bc-4443-8a31-87bc64d2f791 service placement] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:32.468015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22a464-416b-47b0-984d-b01faeb7bb98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:32.468919 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22a464-416b-47b0-984d-b01faeb7bb98 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:32.469512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22a464-416b-47b0-984d-b01faeb7bb98 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:32.469989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22a464-416b-47b0-984d-b01faeb7bb98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:32.471949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fb22a464-416b-47b0-984d-b01faeb7bb98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:32.487088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22a464-416b-47b0-984d-b01faeb7bb98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:32.489147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22a464-416b-47b0-984d-b01faeb7bb98 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:32.495030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:32.501151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1160/2333] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:35:32.531418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0e29f-2eff-4a02-9903-7fc6def58cfa None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:32.538436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:32.539886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:32.540429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:32.541479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] PATH_INFO: `/v3/users/553ec3d9204b4e9baa92839c15587e93` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:32.543466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=553ec3d9204b4e9baa92839c15587e93)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:32.546528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:32.565413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:32.616887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553ec3d9204b4e9baa92839c15587e93'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:32.618924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553ec3d9204b4e9baa92839c15587e93'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:32.630287 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553ec3d9204b4e9baa92839c15587e93'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:32.641314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553ec3d9204b4e9baa92839c15587e93'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:32.642028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553ec3d9204b4e9baa92839c15587e93'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:32.669829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1174/2334] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:35:32 2022] DELETE /identity/v3/users/553ec3d9204b4e9baa92839c15587e93 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:32.674613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34440e-4601-4286-836b-1e941ca2465b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:32.675681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34440e-4601-4286-836b-1e941ca2465b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:32.675681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34440e-4601-4286-836b-1e941ca2465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:32.694935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3e34440e-4601-4286-836b-1e941ca2465b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:32.696814 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3e34440e-4601-4286-836b-1e941ca2465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:32.771916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1161/2335] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:32.788284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:32.789942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:32.886824 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445e9f0-03d2-4201-92aa-c1a7432da0ee admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:32.891063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef513268-fc78-4357-81d4-fb6f233ca45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:32.891681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef513268-fc78-4357-81d4-fb6f233ca45e service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:32.891958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef513268-fc78-4357-81d4-fb6f233ca45e service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:32.892233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef513268-fc78-4357-81d4-fb6f233ca45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:32.903619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef513268-fc78-4357-81d4-fb6f233ca45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:32.905175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef513268-fc78-4357-81d4-fb6f233ca45e service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:32.917355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1175/2336] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:33.530590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34440e-4601-4286-836b-1e941ca2465b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:33.532202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3e34440e-4601-4286-836b-1e941ca2465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:33.546777 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34440e-4601-4286-836b-1e941ca2465b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:33.557801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:33.558732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:33.559225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:33.559594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] PATH_INFO: `/v3/projects/197f8c3a5eb6418c8e0c66d43f6fd958` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:33.561211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197f8c3a5eb6418c8e0c66d43f6fd958)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:33.563253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:33.630738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197f8c3a5eb6418c8e0c66d43f6fd958', 'request_id': 'req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.637630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197f8c3a5eb6418c8e0c66d43f6fd958', 'request_id': 'req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.644662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197f8c3a5eb6418c8e0c66d43f6fd958', 'request_id': 'req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.658480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] The token cache is being invalidate because project 197f8c3a5eb6418c8e0c66d43f6fd958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:35:33.658869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.660213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.662194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1162/2337] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:35:33 2022] DELETE /identity/v3/projects/197f8c3a5eb6418c8e0c66d43f6fd958 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:33.666298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef513268-fc78-4357-81d4-fb6f233ca45e service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:33.677056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef513268-fc78-4357-81d4-fb6f233ca45e service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:33.740128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:33.740801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:33.741095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:33.741371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] PATH_INFO: `/v3/users/91019cd9e99d4a4b856826e190bec0cb` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:33.742513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] RBAC: Authorizing `identity:delete_user(user_id=91019cd9e99d4a4b856826e190bec0cb)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:33.744472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:33.786991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:33.794521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91019cd9e99d4a4b856826e190bec0cb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.795711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91019cd9e99d4a4b856826e190bec0cb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.805035 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91019cd9e99d4a4b856826e190bec0cb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.812376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91019cd9e99d4a4b856826e190bec0cb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.813144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-02b2af94-406a-4fd3-8c2a-48d51eb8c15c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91019cd9e99d4a4b856826e190bec0cb'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:33.833387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1176/2338] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:35:33 2022] DELETE /identity/v3/users/91019cd9e99d4a4b856826e190bec0cb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:34.228059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:34.229613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:34.250110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b8e2b-346b-4db2-aef9-9ce0d574c5c1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:34.344308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:34.345188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.345679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.346074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] PATH_INFO: `/v3/projects/275b335f84e6499a84e149821492cc1f` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.347712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] RBAC: Authorizing `identity:delete_project(project_id=275b335f84e6499a84e149821492cc1f)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:34.349760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:34.384273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:34.406322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275b335f84e6499a84e149821492cc1f', 'request_id': 'req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:34.412270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275b335f84e6499a84e149821492cc1f', 'request_id': 'req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:34.418829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275b335f84e6499a84e149821492cc1f', 'request_id': 'req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:34.433322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] The token cache is being invalidate because project 275b335f84e6499a84e149821492cc1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:35:34.433670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:34.434326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-efe58cd5-3dbb-4705-8f27-ffd89b49d90b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:34.435833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1163/2339] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:35:34 2022] DELETE /identity/v3/projects/275b335f84e6499a84e149821492cc1f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:34.441202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69dc69-b3eb-43ff-9d8b-ee820d8866af None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.441670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69dc69-b3eb-43ff-9d8b-ee820d8866af None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.441951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69dc69-b3eb-43ff-9d8b-ee820d8866af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.461992 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-8e69dc69-b3eb-43ff-9d8b-ee820d8866af None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:34.463201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8e69dc69-b3eb-43ff-9d8b-ee820d8866af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:34.521647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1177/2340] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:34.530375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2abf2-f047-4c96-bdfa-3b6ef2f9e171 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.530654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2abf2-f047-4c96-bdfa-3b6ef2f9e171 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.531219 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2abf2-f047-4c96-bdfa-3b6ef2f9e171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.550592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-14c2abf2-f047-4c96-bdfa-3b6ef2f9e171 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:34.551692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-14c2abf2-f047-4c96-bdfa-3b6ef2f9e171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:34.559283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1164/2341] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:34.563393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69dc69-b3eb-43ff-9d8b-ee820d8866af None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:34.574290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69dc69-b3eb-43ff-9d8b-ee820d8866af None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:34.579143 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29f29a-4235-46b7-b97c-192b1a83af2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:34.579735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29f29a-4235-46b7-b97c-192b1a83af2c admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.580015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29f29a-4235-46b7-b97c-192b1a83af2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.580287 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29f29a-4235-46b7-b97c-192b1a83af2c admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.580973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29f29a-4235-46b7-b97c-192b1a83af2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:35:34.581275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29f29a-4235-46b7-b97c-192b1a83af2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:34.583945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29f29a-4235-46b7-b97c-192b1a83af2c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:34.595392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1178/2342] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:35:34.599654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2abf2-f047-4c96-bdfa-3b6ef2f9e171 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:34.608618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2abf2-f047-4c96-bdfa-3b6ef2f9e171 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:34.613156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67a6b6-9d3f-4b93-b7be-a1fe8382d022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:34.613757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67a6b6-9d3f-4b93-b7be-a1fe8382d022 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.614029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67a6b6-9d3f-4b93-b7be-a1fe8382d022 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.614293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67a6b6-9d3f-4b93-b7be-a1fe8382d022 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.614908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67a6b6-9d3f-4b93-b7be-a1fe8382d022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:34.616841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67a6b6-9d3f-4b93-b7be-a1fe8382d022 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:34.642964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1165/2343] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:35:34 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:34.647808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29f29a-4235-46b7-b97c-192b1a83af2c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:34.657735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29f29a-4235-46b7-b97c-192b1a83af2c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:34.714252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccec67-e4fb-4b0c-9c3d-cf33d5feafa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:34.714892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccec67-e4fb-4b0c-9c3d-cf33d5feafa4 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.715120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccec67-e4fb-4b0c-9c3d-cf33d5feafa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.715400 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccec67-e4fb-4b0c-9c3d-cf33d5feafa4 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.716048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccec67-e4fb-4b0c-9c3d-cf33d5feafa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:34.718060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccec67-e4fb-4b0c-9c3d-cf33d5feafa4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:34.739805 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1179/2344] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:35:34 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:34.743955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67a6b6-9d3f-4b93-b7be-a1fe8382d022 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:34.754544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67a6b6-9d3f-4b93-b7be-a1fe8382d022 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:34.759030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b91e4-0126-472c-b08b-22d6b513af5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:34.759615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b91e4-0126-472c-b08b-22d6b513af5d admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.759887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b91e4-0126-472c-b08b-22d6b513af5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.760155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b91e4-0126-472c-b08b-22d6b513af5d admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.760837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b91e4-0126-472c-b08b-22d6b513af5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:34.762424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b91e4-0126-472c-b08b-22d6b513af5d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:34.772596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1166/2345] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:34.776374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccec67-e4fb-4b0c-9c3d-cf33d5feafa4 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:34.786355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccec67-e4fb-4b0c-9c3d-cf33d5feafa4 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:34.790965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3b09d-1acf-4dd2-97e7-09c4cc5bf637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:34.791561 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3b09d-1acf-4dd2-97e7-09c4cc5bf637 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.791845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3b09d-1acf-4dd2-97e7-09c4cc5bf637 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.792118 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3b09d-1acf-4dd2-97e7-09c4cc5bf637 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.792843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3b09d-1acf-4dd2-97e7-09c4cc5bf637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:34.794450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3b09d-1acf-4dd2-97e7-09c4cc5bf637 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:34.805730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1180/2346] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:34.809727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b91e4-0126-472c-b08b-22d6b513af5d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:34.820210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b91e4-0126-472c-b08b-22d6b513af5d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:34.825015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a267bc3-07bc-4b54-adcf-1c53e94f9bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:34.825657 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a267bc3-07bc-4b54-adcf-1c53e94f9bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.826194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a267bc3-07bc-4b54-adcf-1c53e94f9bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.826466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a267bc3-07bc-4b54-adcf-1c53e94f9bb9 admin admin] PATH_INFO: `/v3/projects/f304ab61c8c140b38bf828d0d2930f4d/users/58f8e6139de647cb8477dd39407191e7/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.841359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a267bc3-07bc-4b54-adcf-1c53e94f9bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f304ab61c8c140b38bf828d0d2930f4d, user_id=58f8e6139de647cb8477dd39407191e7, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:34.844009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a267bc3-07bc-4b54-adcf-1c53e94f9bb9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:34.857640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1167/2347] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:35:34 2022] PUT /identity/v3/projects/f304ab61c8c140b38bf828d0d2930f4d/users/58f8e6139de647cb8477dd39407191e7/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:34.863207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecc36b-249a-452a-bc77-e2b07ba0c8e3 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:34.863514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecc36b-249a-452a-bc77-e2b07ba0c8e3 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:34.863968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ecc36b-249a-452a-bc77-e2b07ba0c8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:34.885926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-43ecc36b-249a-452a-bc77-e2b07ba0c8e3 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:34.887144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-43ecc36b-249a-452a-bc77-e2b07ba0c8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:34.904781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:34.950038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1181/2348] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:34.960595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a267bc3-07bc-4b54-adcf-1c53e94f9bb9 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:34.962288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5a267bc3-07bc-4b54-adcf-1c53e94f9bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:35.020873 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a267bc3-07bc-4b54-adcf-1c53e94f9bb9 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:35.026307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f072e-1289-4599-87da-b77bf354a755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:35.026918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f072e-1289-4599-87da-b77bf354a755 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:35.027194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f072e-1289-4599-87da-b77bf354a755 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:35.027461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f072e-1289-4599-87da-b77bf354a755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:35.037826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f072e-1289-4599-87da-b77bf354a755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:35.039668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f072e-1289-4599-87da-b77bf354a755 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:35.050332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1168/2349] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:35.764600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecc36b-249a-452a-bc77-e2b07ba0c8e3 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:35.766053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-43ecc36b-249a-452a-bc77-e2b07ba0c8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:35.858928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ecc36b-249a-452a-bc77-e2b07ba0c8e3 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:35.865895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48275bb9-1de7-4466-83c2-4a0dc090427e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:35.866678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48275bb9-1de7-4466-83c2-4a0dc090427e service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:35.866884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48275bb9-1de7-4466-83c2-4a0dc090427e service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:35.867199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48275bb9-1de7-4466-83c2-4a0dc090427e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:35.869528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-48275bb9-1de7-4466-83c2-4a0dc090427e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:35.886321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48275bb9-1de7-4466-83c2-4a0dc090427e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:35.888475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48275bb9-1de7-4466-83c2-4a0dc090427e service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:35.901269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1182/2350] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:35.941064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46565579-07b3-43ec-96cb-bcfa36f6abfe None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:35.941706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46565579-07b3-43ec-96cb-bcfa36f6abfe None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:35.941706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46565579-07b3-43ec-96cb-bcfa36f6abfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:35.983879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-46565579-07b3-43ec-96cb-bcfa36f6abfe None None] MFA Rules not processed for user `58f8e6139de647cb8477dd39407191e7`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:35.985182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-46565579-07b3-43ec-96cb-bcfa36f6abfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:36.039592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:36.076138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1169/2351] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:36.080465 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48275bb9-1de7-4466-83c2-4a0dc090427e service nova] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:36.092353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48275bb9-1de7-4466-83c2-4a0dc090427e service nova] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:36.108746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba5455-6977-4298-a324-de8b8085b0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:36.109507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba5455-6977-4298-a324-de8b8085b0db admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:36.110466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba5455-6977-4298-a324-de8b8085b0db admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:36.110826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba5455-6977-4298-a324-de8b8085b0db admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:36.111755 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba5455-6977-4298-a324-de8b8085b0db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:36.118508 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba5455-6977-4298-a324-de8b8085b0db admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:36.165291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1183/2352] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:35:36 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:36.169436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46565579-07b3-43ec-96cb-bcfa36f6abfe None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:36.184478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46565579-07b3-43ec-96cb-bcfa36f6abfe None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:36.263366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662157d-5ea0-448f-823e-8ac5c5a11854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:36.264047 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662157d-5ea0-448f-823e-8ac5c5a11854 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:36.264599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662157d-5ea0-448f-823e-8ac5c5a11854 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:36.264946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662157d-5ea0-448f-823e-8ac5c5a11854 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:36.265589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662157d-5ea0-448f-823e-8ac5c5a11854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:36.267493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662157d-5ea0-448f-823e-8ac5c5a11854 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:36.289756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1170/2353] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:35:36 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:35:36.295371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba5455-6977-4298-a324-de8b8085b0db admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:36.306151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba5455-6977-4298-a324-de8b8085b0db admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:36.311074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19cd1c9-ed69-40f5-a9cc-843a7d98e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:36.311750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cd1c9-ed69-40f5-a9cc-843a7d98e6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:36.312152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cd1c9-ed69-40f5-a9cc-843a7d98e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:36.312365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cd1c9-ed69-40f5-a9cc-843a7d98e6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:36.313298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19cd1c9-ed69-40f5-a9cc-843a7d98e6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:36.315004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19cd1c9-ed69-40f5-a9cc-843a7d98e6e6 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:36.326881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1184/2354] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:36.330913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662157d-5ea0-448f-823e-8ac5c5a11854 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:36.340785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662157d-5ea0-448f-823e-8ac5c5a11854 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:36.346036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bb323-2ee5-4ca4-b784-ee18281de960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:36.347197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bb323-2ee5-4ca4-b784-ee18281de960 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:36.347301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bb323-2ee5-4ca4-b784-ee18281de960 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:36.347813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bb323-2ee5-4ca4-b784-ee18281de960 admin admin] PATH_INFO: `/v3/projects/167809ca76fb4c7a9f36db2731735c7a/users/dcc423d7cd2d49b499c9d73afb178818/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:36.364851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bb323-2ee5-4ca4-b784-ee18281de960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=167809ca76fb4c7a9f36db2731735c7a, user_id=dcc423d7cd2d49b499c9d73afb178818, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:36.367611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bb323-2ee5-4ca4-b784-ee18281de960 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:36.383490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1171/2355] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:35:36 2022] PUT /identity/v3/projects/167809ca76fb4c7a9f36db2731735c7a/users/dcc423d7cd2d49b499c9d73afb178818/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:36.387674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19cd1c9-ed69-40f5-a9cc-843a7d98e6e6 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:36.398398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19cd1c9-ed69-40f5-a9cc-843a7d98e6e6 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:36.457595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e724-9e60-4ed6-ad71-1ac777d24126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:36.458201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e724-9e60-4ed6-ad71-1ac777d24126 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:36.458480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e724-9e60-4ed6-ad71-1ac777d24126 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:36.458751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e724-9e60-4ed6-ad71-1ac777d24126 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:36.459383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154e724-9e60-4ed6-ad71-1ac777d24126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:36.461013 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154e724-9e60-4ed6-ad71-1ac777d24126 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:36.475173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1185/2356] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:35:36.479196 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bb323-2ee5-4ca4-b784-ee18281de960 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:36.489398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bb323-2ee5-4ca4-b784-ee18281de960 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:36.494148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db43ae-8534-44f1-a9d1-5247563127a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:36.494798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db43ae-8534-44f1-a9d1-5247563127a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:36.495069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db43ae-8534-44f1-a9d1-5247563127a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:36.495336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db43ae-8534-44f1-a9d1-5247563127a7 admin admin] PATH_INFO: `/v3/projects/167809ca76fb4c7a9f36db2731735c7a/users/dcc423d7cd2d49b499c9d73afb178818/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:36.497245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db43ae-8534-44f1-a9d1-5247563127a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=167809ca76fb4c7a9f36db2731735c7a, user_id=dcc423d7cd2d49b499c9d73afb178818, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:36.499868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db43ae-8534-44f1-a9d1-5247563127a7 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:36.510933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1172/2357] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:35:36 2022] PUT /identity/v3/projects/167809ca76fb4c7a9f36db2731735c7a/users/dcc423d7cd2d49b499c9d73afb178818/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:35:36.952933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7822df-b1e6-43c0-9d93-0dd3c1314d8f None None] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:36.953420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7822df-b1e6-43c0-9d93-0dd3c1314d8f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:36.953544 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7822df-b1e6-43c0-9d93-0dd3c1314d8f None None] PATH_INFO: `/` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:36.954788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1186/2358] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 1 00:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 00:35:36.959961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80132c-77e4-49e6-a8f2-532b09aeb875 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:36.960483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80132c-77e4-49e6-a8f2-532b09aeb875 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:36.960759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80132c-77e4-49e6-a8f2-532b09aeb875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:36.981683 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-4c80132c-77e4-49e6-a8f2-532b09aeb875 None None] MFA Rules not processed for user `fb204040ab2f4168b35c3e78bacb849f`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:36.983130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4c80132c-77e4-49e6-a8f2-532b09aeb875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:37.079299 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1173/2359] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 1 00:35:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:35:37.091493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7822df-b1e6-43c0-9d93-0dd3c1314d8f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:37.093227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7b7822df-b1e6-43c0-9d93-0dd3c1314d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:37.108170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:37.171189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7822df-b1e6-43c0-9d93-0dd3c1314d8f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:37.176210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92e77-6638-425d-8720-72e14a7f8cf2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52470b620abd4cd29ab7e4752f285bb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:37.176895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92e77-6638-425d-8720-72e14a7f8cf2 service glance] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:37.177181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92e77-6638-425d-8720-72e14a7f8cf2 service glance] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:37.177458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92e77-6638-425d-8720-72e14a7f8cf2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:37.188380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b92e77-6638-425d-8720-72e14a7f8cf2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:37.189857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b92e77-6638-425d-8720-72e14a7f8cf2 service glance] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:37.203803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1187/2360] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:37.291654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c38380-6f4b-4cf4-826a-e236c3a23f8a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:37.292223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c38380-6f4b-4cf4-826a-e236c3a23f8a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:37.292529 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c38380-6f4b-4cf4-826a-e236c3a23f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:37.326041 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-36c38380-6f4b-4cf4-826a-e236c3a23f8a None None] MFA Rules not processed for user `dcc423d7cd2d49b499c9d73afb178818`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:37.327884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-36c38380-6f4b-4cf4-826a-e236c3a23f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:37.344929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:37.441065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1174/2361] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:37.452102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92e77-6638-425d-8720-72e14a7f8cf2 service glance] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:37.453553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-64b92e77-6638-425d-8720-72e14a7f8cf2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:37.516754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92e77-6638-425d-8720-72e14a7f8cf2 service glance] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:37.521185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad378d-3918-4ace-b857-9b8cb4f2d981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:37.521837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad378d-3918-4ace-b857-9b8cb4f2d981 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:37.522117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad378d-3918-4ace-b857-9b8cb4f2d981 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:37.522393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad378d-3918-4ace-b857-9b8cb4f2d981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:37.580346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad378d-3918-4ace-b857-9b8cb4f2d981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:37.582888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad378d-3918-4ace-b857-9b8cb4f2d981 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:37.592140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1188/2362] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:39.011775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c38380-6f4b-4cf4-826a-e236c3a23f8a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:39.013300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-36c38380-6f4b-4cf4-826a-e236c3a23f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:39.027276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c38380-6f4b-4cf4-826a-e236c3a23f8a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:39.031486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c87b4-352d-4a29-ad5d-9fb99eb949ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:39.032093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c87b4-352d-4a29-ad5d-9fb99eb949ce service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:39.032370 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c87b4-352d-4a29-ad5d-9fb99eb949ce service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:39.032687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c87b4-352d-4a29-ad5d-9fb99eb949ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:39.042941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c87b4-352d-4a29-ad5d-9fb99eb949ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:39.044472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c87b4-352d-4a29-ad5d-9fb99eb949ce service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:39.057419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1175/2363] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:40.449465 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765d616-9b00-41ed-ae2e-f07e0396da95 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:40.449933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765d616-9b00-41ed-ae2e-f07e0396da95 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:40.450028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765d616-9b00-41ed-ae2e-f07e0396da95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:40.484511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-c765d616-9b00-41ed-ae2e-f07e0396da95 None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:40.485933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c765d616-9b00-41ed-ae2e-f07e0396da95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:40.556480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1189/2364] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:35:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:40.570775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c87b4-352d-4a29-ad5d-9fb99eb949ce service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:40.572213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-287c87b4-352d-4a29-ad5d-9fb99eb949ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:40.725632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c87b4-352d-4a29-ad5d-9fb99eb949ce service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:40.739920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5011bc3-cb87-4fcf-a150-03ddf79e3655 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:40.740380 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5011bc3-cb87-4fcf-a150-03ddf79e3655 service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:40.740887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5011bc3-cb87-4fcf-a150-03ddf79e3655 service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:40.741364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5011bc3-cb87-4fcf-a150-03ddf79e3655 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:40.769964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5011bc3-cb87-4fcf-a150-03ddf79e3655 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:40.772000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5011bc3-cb87-4fcf-a150-03ddf79e3655 service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:40.783783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1176/2365] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:35:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:35:46.166711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765d616-9b00-41ed-ae2e-f07e0396da95 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:46.201654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:46.271552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765d616-9b00-41ed-ae2e-f07e0396da95 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:46.277142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:46.278525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:46.278951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:46.279550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:46.287290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:46.376951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:46.381801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:46.401118 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1190/2366] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:50.591311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5011bc3-cb87-4fcf-a150-03ddf79e3655 service swift] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:50.592900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f5011bc3-cb87-4fcf-a150-03ddf79e3655 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:50.661001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:50.663834 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:50.693701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e904954-829e-46ae-8494-8526bcbc89fc service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:50.696279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5011bc3-cb87-4fcf-a150-03ddf79e3655 service swift] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:50.702045 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7862f11-781f-4e09-8cd9-d96c10dd1da0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:50.702726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7862f11-781f-4e09-8cd9-d96c10dd1da0 service placement] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:50.703291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7862f11-781f-4e09-8cd9-d96c10dd1da0 service placement] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:50.703636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7862f11-781f-4e09-8cd9-d96c10dd1da0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:50.705086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:50.706740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:50.706868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a7862f11-781f-4e09-8cd9-d96c10dd1da0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:50.707365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:50.707851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] PATH_INFO: `/v3/users/5bf265cb4af14e2d8815c42288e2a1f7` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:50.717397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf265cb4af14e2d8815c42288e2a1f7)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:50.719904 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:50.729711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7862f11-781f-4e09-8cd9-d96c10dd1da0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:50.731512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7862f11-781f-4e09-8cd9-d96c10dd1da0 service placement] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:50.748631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1177/2367] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:50.790783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf265cb4af14e2d8815c42288e2a1f7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:50.791698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf265cb4af14e2d8815c42288e2a1f7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:50.803072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf265cb4af14e2d8815c42288e2a1f7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:50.813094 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf265cb4af14e2d8815c42288e2a1f7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:50.813902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-3c437072-f886-4475-93a6-05f84c4283f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf265cb4af14e2d8815c42288e2a1f7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:50.849017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1191/2368] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:35:50 2022] DELETE /identity/v3/users/5bf265cb4af14e2d8815c42288e2a1f7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:51.195142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7862f11-781f-4e09-8cd9-d96c10dd1da0 service placement] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:51.206724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7862f11-781f-4e09-8cd9-d96c10dd1da0 service placement] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:51.223069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:51.270409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:51.271038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.271315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.271586 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] PATH_INFO: `/v3/projects/f1064fc209674020b54fa7526cf22bf5` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.272748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1064fc209674020b54fa7526cf22bf5)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:51.274654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:51.321315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1064fc209674020b54fa7526cf22bf5', 'request_id': 'req-bf889474-7b95-4606-9054-a2aabf3fc947'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:51.327030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1064fc209674020b54fa7526cf22bf5', 'request_id': 'req-bf889474-7b95-4606-9054-a2aabf3fc947'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:51.333965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1064fc209674020b54fa7526cf22bf5', 'request_id': 'req-bf889474-7b95-4606-9054-a2aabf3fc947'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:51.349114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] The token cache is being invalidate because project f1064fc209674020b54fa7526cf22bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:35:51.349402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:51.350033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bf889474-7b95-4606-9054-a2aabf3fc947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:51.351525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1178/2369] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:35:51 2022] DELETE /identity/v3/projects/f1064fc209674020b54fa7526cf22bf5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:51.357148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcc683-0398-4cf6-b883-ac4635618b32 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.357701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcc683-0398-4cf6-b883-ac4635618b32 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.358059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fcc683-0398-4cf6-b883-ac4635618b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.379822 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-c5fcc683-0398-4cf6-b883-ac4635618b32 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:51.381051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c5fcc683-0398-4cf6-b883-ac4635618b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:51.414322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:51.447121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1192/2370] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:51.455246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f28dd-30d7-4929-9829-e1de42c6874a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.456482 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f28dd-30d7-4929-9829-e1de42c6874a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.456860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f28dd-30d7-4929-9829-e1de42c6874a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.480415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e44f28dd-30d7-4929-9829-e1de42c6874a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:51.481677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e44f28dd-30d7-4929-9829-e1de42c6874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:51.489694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1179/2371] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:51.494042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcc683-0398-4cf6-b883-ac4635618b32 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:51.505199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fcc683-0398-4cf6-b883-ac4635618b32 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:51.509876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757a41c-206f-4ae0-ae6d-598aea65d210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:51.510479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757a41c-206f-4ae0-ae6d-598aea65d210 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.510755 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757a41c-206f-4ae0-ae6d-598aea65d210 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.511023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757a41c-206f-4ae0-ae6d-598aea65d210 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.511672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757a41c-206f-4ae0-ae6d-598aea65d210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:35:51.511971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757a41c-206f-4ae0-ae6d-598aea65d210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:51.513697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757a41c-206f-4ae0-ae6d-598aea65d210 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:51.527006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1193/2372] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:35:51.531470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f28dd-30d7-4929-9829-e1de42c6874a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:51.541269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f28dd-30d7-4929-9829-e1de42c6874a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:51.545902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084da5a-64fb-4908-8ea8-3fe2f2f2104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:51.546521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084da5a-64fb-4908-8ea8-3fe2f2f2104e admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.546800 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084da5a-64fb-4908-8ea8-3fe2f2f2104e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.547138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084da5a-64fb-4908-8ea8-3fe2f2f2104e admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.547724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084da5a-64fb-4908-8ea8-3fe2f2f2104e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:51.549962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084da5a-64fb-4908-8ea8-3fe2f2f2104e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:51.581728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1180/2373] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:35:51 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:51.586179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757a41c-206f-4ae0-ae6d-598aea65d210 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:51.596889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757a41c-206f-4ae0-ae6d-598aea65d210 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:51.662369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcfdca-2afb-487b-abd0-193d606a75ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:51.663213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcfdca-2afb-487b-abd0-193d606a75ab admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.663493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcfdca-2afb-487b-abd0-193d606a75ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.663764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcfdca-2afb-487b-abd0-193d606a75ab admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.664382 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcfdca-2afb-487b-abd0-193d606a75ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:51.666368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcfdca-2afb-487b-abd0-193d606a75ab admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:51.688458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1194/2374] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:35:51 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:51.693093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084da5a-64fb-4908-8ea8-3fe2f2f2104e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:51.703515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084da5a-64fb-4908-8ea8-3fe2f2f2104e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:51.708953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df2ff3-e38b-4d4f-97e7-a6acc02df4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:51.709550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df2ff3-e38b-4d4f-97e7-a6acc02df4cf admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.709827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df2ff3-e38b-4d4f-97e7-a6acc02df4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.710099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df2ff3-e38b-4d4f-97e7-a6acc02df4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.710739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df2ff3-e38b-4d4f-97e7-a6acc02df4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:51.712348 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df2ff3-e38b-4d4f-97e7-a6acc02df4cf admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:51.722378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1181/2375] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:51.726324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcfdca-2afb-487b-abd0-193d606a75ab admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:51.736821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcfdca-2afb-487b-abd0-193d606a75ab admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:51.741595 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c4b4b-610b-47ad-ae47-ccc1d9abf9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:51.742191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c4b4b-610b-47ad-ae47-ccc1d9abf9bb admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.742469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c4b4b-610b-47ad-ae47-ccc1d9abf9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.742740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c4b4b-610b-47ad-ae47-ccc1d9abf9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.743380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c4b4b-610b-47ad-ae47-ccc1d9abf9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:51.745001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c4b4b-610b-47ad-ae47-ccc1d9abf9bb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:51.756814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1195/2376] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:51.760900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df2ff3-e38b-4d4f-97e7-a6acc02df4cf admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:51.771432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df2ff3-e38b-4d4f-97e7-a6acc02df4cf admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:51.776073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f37e-3143-435b-a4f3-e79e5f035cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:51.776745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f37e-3143-435b-a4f3-e79e5f035cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.777025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f37e-3143-435b-a4f3-e79e5f035cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.777296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f37e-3143-435b-a4f3-e79e5f035cb9 admin admin] PATH_INFO: `/v3/projects/fffe55914dca4883a2a77a2b017e8cde/users/0a6e3415ba5d491c955a9d4e44c99e4b/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.792166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3f37e-3143-435b-a4f3-e79e5f035cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fffe55914dca4883a2a77a2b017e8cde, user_id=0a6e3415ba5d491c955a9d4e44c99e4b, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:51.794854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3f37e-3143-435b-a4f3-e79e5f035cb9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:51.807734 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1182/2377] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:35:51 2022] PUT /identity/v3/projects/fffe55914dca4883a2a77a2b017e8cde/users/0a6e3415ba5d491c955a9d4e44c99e4b/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:51.819826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb0228-1530-44ec-8676-0078ae809a1a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:51.820125 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb0228-1530-44ec-8676-0078ae809a1a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:51.820386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb0228-1530-44ec-8676-0078ae809a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:51.849274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bcbb0228-1530-44ec-8676-0078ae809a1a None None] MFA Rules not processed for user `0a6e3415ba5d491c955a9d4e44c99e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:51.850396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bcbb0228-1530-44ec-8676-0078ae809a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:51.918713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1196/2378] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:51.932347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f37e-3143-435b-a4f3-e79e5f035cb9 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:51.934123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-87a3f37e-3143-435b-a4f3-e79e5f035cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:51.982329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:52.012983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f37e-3143-435b-a4f3-e79e5f035cb9 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:52.018437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65dd3a-5a5c-437d-bd83-cf53f1b39c84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:52.019037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65dd3a-5a5c-437d-bd83-cf53f1b39c84 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:52.019313 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65dd3a-5a5c-437d-bd83-cf53f1b39c84 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:52.019578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65dd3a-5a5c-437d-bd83-cf53f1b39c84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:52.027839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65dd3a-5a5c-437d-bd83-cf53f1b39c84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:52.029973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65dd3a-5a5c-437d-bd83-cf53f1b39c84 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:52.039261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1183/2379] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:52.892342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb0228-1530-44ec-8676-0078ae809a1a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:52.893829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bcbb0228-1530-44ec-8676-0078ae809a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:52.929280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:52.981100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb0228-1530-44ec-8676-0078ae809a1a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:52.986175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000f309-f626-413f-b581-cb804845f209 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:52.987469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000f309-f626-413f-b581-cb804845f209 service placement] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:52.987748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000f309-f626-413f-b581-cb804845f209 service placement] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:52.988021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000f309-f626-413f-b581-cb804845f209 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:52.989823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d000f309-f626-413f-b581-cb804845f209 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:53.001639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000f309-f626-413f-b581-cb804845f209 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:53.003925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000f309-f626-413f-b581-cb804845f209 service placement] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:53.014085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1197/2380] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:53.626645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65dd3a-5a5c-437d-bd83-cf53f1b39c84 service nova] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:53.628197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ce65dd3a-5a5c-437d-bd83-cf53f1b39c84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:53.726899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65dd3a-5a5c-437d-bd83-cf53f1b39c84 service nova] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:53.733400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:53.734050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:53.734376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:53.734836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] PATH_INFO: `/v3/users/58f8e6139de647cb8477dd39407191e7` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:53.736395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f8e6139de647cb8477dd39407191e7)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:53.739570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:53.800968 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f8e6139de647cb8477dd39407191e7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:53.801768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f8e6139de647cb8477dd39407191e7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:53.818203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f8e6139de647cb8477dd39407191e7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:53.826327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f8e6139de647cb8477dd39407191e7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:53.834991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f8e6139de647cb8477dd39407191e7'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:53.842056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1184/2381] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:35:53 2022] DELETE /identity/v3/users/58f8e6139de647cb8477dd39407191e7 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:54.266661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000f309-f626-413f-b581-cb804845f209 service placement] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:54.278142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000f309-f626-413f-b581-cb804845f209 service placement] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:54.363720 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:54.364423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:54.364798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:54.365423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] PATH_INFO: `/v3/projects/f304ab61c8c140b38bf828d0d2930f4d` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:54.366761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f304ab61c8c140b38bf828d0d2930f4d)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:54.368771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:54.417599 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f304ab61c8c140b38bf828d0d2930f4d', 'request_id': 'req-234c74ce-ee72-41d6-8767-a810a981e60c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.423422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f304ab61c8c140b38bf828d0d2930f4d', 'request_id': 'req-234c74ce-ee72-41d6-8767-a810a981e60c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.430498 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f304ab61c8c140b38bf828d0d2930f4d', 'request_id': 'req-234c74ce-ee72-41d6-8767-a810a981e60c'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.444114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] The token cache is being invalidate because project f304ab61c8c140b38bf828d0d2930f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:35:54.444408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.445054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.446532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1198/2382] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:35:54 2022] DELETE /identity/v3/projects/f304ab61c8c140b38bf828d0d2930f4d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:35:54.450887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:54.452368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:54.466624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267370-918a-4bcd-acc4-f49ad2991e6b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:54.513092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:54.526690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:54.527426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:54.527776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:54.528057 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] PATH_INFO: `/v3/users/dcc423d7cd2d49b499c9d73afb178818` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:54.529228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcc423d7cd2d49b499c9d73afb178818)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:54.531215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:54.577816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcc423d7cd2d49b499c9d73afb178818'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.578461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcc423d7cd2d49b499c9d73afb178818'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.590869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcc423d7cd2d49b499c9d73afb178818'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.598404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcc423d7cd2d49b499c9d73afb178818'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.609688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-a9d5c1d1-bd35-4767-8640-e5d6e689c0c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcc423d7cd2d49b499c9d73afb178818'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.619430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1185/2383] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:35:54 2022] DELETE /identity/v3/users/dcc423d7cd2d49b499c9d73afb178818 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:35:54.803743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:54.805654 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:54.823704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c74ce-ee72-41d6-8767-a810a981e60c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:54.885874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:54.898611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:54.899273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:54.899553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:54.899825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] PATH_INFO: `/v3/projects/167809ca76fb4c7a9f36db2731735c7a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:54.900986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=167809ca76fb4c7a9f36db2731735c7a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:54.902905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:54.951954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '167809ca76fb4c7a9f36db2731735c7a', 'request_id': 'req-b8218df2-e4de-4957-816c-578c9134c544'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.958273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '167809ca76fb4c7a9f36db2731735c7a', 'request_id': 'req-b8218df2-e4de-4957-816c-578c9134c544'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.964607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '167809ca76fb4c7a9f36db2731735c7a', 'request_id': 'req-b8218df2-e4de-4957-816c-578c9134c544'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.979967 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] The token cache is being invalidate because project 167809ca76fb4c7a9f36db2731735c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:35:54.980263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.981166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b8218df2-e4de-4957-816c-578c9134c544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:54.982964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1199/2384] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:35:54 2022] DELETE /identity/v3/projects/167809ca76fb4c7a9f36db2731735c7a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:54.988014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ab52f-c6e7-4c56-a7f5-deb241ac0cd4 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:54.988551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ab52f-c6e7-4c56-a7f5-deb241ac0cd4 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:54.988551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ab52f-c6e7-4c56-a7f5-deb241ac0cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.007797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2c1ab52f-c6e7-4c56-a7f5-deb241ac0cd4 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:55.009109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2c1ab52f-c6e7-4c56-a7f5-deb241ac0cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:55.073572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1186/2385] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:55.083178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b947f1-da3a-4fc5-a23c-5d92ad09a90b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.083475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b947f1-da3a-4fc5-a23c-5d92ad09a90b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.083756 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b947f1-da3a-4fc5-a23c-5d92ad09a90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.107360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-19b947f1-da3a-4fc5-a23c-5d92ad09a90b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:55.108731 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-19b947f1-da3a-4fc5-a23c-5d92ad09a90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:55.121908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1200/2386] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:55.126211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ab52f-c6e7-4c56-a7f5-deb241ac0cd4 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:55.137595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ab52f-c6e7-4c56-a7f5-deb241ac0cd4 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:55.142363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334aa734-78fa-4899-9953-237de46fab82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:55.145210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334aa734-78fa-4899-9953-237de46fab82 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.145502 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334aa734-78fa-4899-9953-237de46fab82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.145780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334aa734-78fa-4899-9953-237de46fab82 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.146461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334aa734-78fa-4899-9953-237de46fab82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:35:55.146768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334aa734-78fa-4899-9953-237de46fab82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:55.148419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334aa734-78fa-4899-9953-237de46fab82 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:55.162027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1187/2387] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:35:55.166835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b947f1-da3a-4fc5-a23c-5d92ad09a90b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:55.177705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b947f1-da3a-4fc5-a23c-5d92ad09a90b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:55.182544 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b9149-ff05-4d58-97dc-8f6190a48bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:55.183283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9149-ff05-4d58-97dc-8f6190a48bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.183629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9149-ff05-4d58-97dc-8f6190a48bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.183972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9149-ff05-4d58-97dc-8f6190a48bf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.184775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b9149-ff05-4d58-97dc-8f6190a48bf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:55.186710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b9149-ff05-4d58-97dc-8f6190a48bf9 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:55.220003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1201/2388] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:35:55 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:35:55.221250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334aa734-78fa-4899-9953-237de46fab82 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:55.232152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334aa734-78fa-4899-9953-237de46fab82 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:55.298599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:55.307786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe3def-072f-4756-b5c4-5b2421360c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:55.308401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe3def-072f-4756-b5c4-5b2421360c21 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.308881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe3def-072f-4756-b5c4-5b2421360c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.309413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe3def-072f-4756-b5c4-5b2421360c21 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.310305 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe3def-072f-4756-b5c4-5b2421360c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:55.312470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe3def-072f-4756-b5c4-5b2421360c21 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:55.335870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1188/2389] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:35:55 2022] POST /identity/v3/users => generated 281 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:55.340155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b9149-ff05-4d58-97dc-8f6190a48bf9 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:55.351336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b9149-ff05-4d58-97dc-8f6190a48bf9 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:55.355995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa194e-01b7-469f-90e5-0b5fecc3aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:55.356639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa194e-01b7-469f-90e5-0b5fecc3aec2 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.356935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa194e-01b7-469f-90e5-0b5fecc3aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.357198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa194e-01b7-469f-90e5-0b5fecc3aec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.357840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa194e-01b7-469f-90e5-0b5fecc3aec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:55.359489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa194e-01b7-469f-90e5-0b5fecc3aec2 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:55.371098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1202/2390] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:55.375167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe3def-072f-4756-b5c4-5b2421360c21 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:55.385878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe3def-072f-4756-b5c4-5b2421360c21 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:55.390648 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cd76d-e158-4c36-a41d-7db52d722297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:55.391254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cd76d-e158-4c36-a41d-7db52d722297 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.391533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cd76d-e158-4c36-a41d-7db52d722297 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.391804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cd76d-e158-4c36-a41d-7db52d722297 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.392446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cd76d-e158-4c36-a41d-7db52d722297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:55.394537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cd76d-e158-4c36-a41d-7db52d722297 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:55.405308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1189/2391] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:55.410154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa194e-01b7-469f-90e5-0b5fecc3aec2 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:55.421237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa194e-01b7-469f-90e5-0b5fecc3aec2 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:55.427259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699b39a-3ba7-45a3-bc85-2043b98711ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:55.427968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699b39a-3ba7-45a3-bc85-2043b98711ff admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.428320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699b39a-3ba7-45a3-bc85-2043b98711ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.428700 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699b39a-3ba7-45a3-bc85-2043b98711ff admin admin] PATH_INFO: `/v3/projects/b5405e58b6da40edb1456103cac516d3/users/d9384dd9cea9412f91b34b5a7f690fb7/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.446594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699b39a-3ba7-45a3-bc85-2043b98711ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5405e58b6da40edb1456103cac516d3, user_id=d9384dd9cea9412f91b34b5a7f690fb7, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:55.449431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699b39a-3ba7-45a3-bc85-2043b98711ff admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:55.465540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1203/2392] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:35:55 2022] PUT /identity/v3/projects/b5405e58b6da40edb1456103cac516d3/users/d9384dd9cea9412f91b34b5a7f690fb7/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:55.477038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1242a13-f43e-46c6-94af-d2d1797be9fd None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.477330 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1242a13-f43e-46c6-94af-d2d1797be9fd None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.477611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1242a13-f43e-46c6-94af-d2d1797be9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.506464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699b39a-3ba7-45a3-bc85-2043b98711ff admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:55.508338 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b699b39a-3ba7-45a3-bc85-2043b98711ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:55.510765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-a1242a13-f43e-46c6-94af-d2d1797be9fd None None] MFA Rules not processed for user `d9384dd9cea9412f91b34b5a7f690fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:55.511855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a1242a13-f43e-46c6-94af-d2d1797be9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:55.594945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1190/2393] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:55.611835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699b39a-3ba7-45a3-bc85-2043b98711ff admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:55.615656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad28cac-fd57-4903-8bee-be4ab7f95fe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:55.616160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad28cac-fd57-4903-8bee-be4ab7f95fe2 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.616482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad28cac-fd57-4903-8bee-be4ab7f95fe2 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.616736 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad28cac-fd57-4903-8bee-be4ab7f95fe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.621840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8ad28cac-fd57-4903-8bee-be4ab7f95fe2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:55.631211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1242a13-f43e-46c6-94af-d2d1797be9fd None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:55.632766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a1242a13-f43e-46c6-94af-d2d1797be9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:55.636621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad28cac-fd57-4903-8bee-be4ab7f95fe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:55.638292 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad28cac-fd57-4903-8bee-be4ab7f95fe2 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:55.651223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1204/2394] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:35:55.701009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1242a13-f43e-46c6-94af-d2d1797be9fd None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:55.704574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498544a4-14fd-43d5-8675-6965cdaedf66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:55.705235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498544a4-14fd-43d5-8675-6965cdaedf66 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.705558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498544a4-14fd-43d5-8675-6965cdaedf66 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.706079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498544a4-14fd-43d5-8675-6965cdaedf66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.714177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498544a4-14fd-43d5-8675-6965cdaedf66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:55.715639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498544a4-14fd-43d5-8675-6965cdaedf66 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:55.725261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1191/2395] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:35:55.741084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad28cac-fd57-4903-8bee-be4ab7f95fe2 service nova] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:55.745556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:55.807614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad28cac-fd57-4903-8bee-be4ab7f95fe2 service nova] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:55.812766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:55.813397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.813676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.813946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] PATH_INFO: `/v3/users/d9384dd9cea9412f91b34b5a7f690fb7` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.815666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9384dd9cea9412f91b34b5a7f690fb7)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:55.817630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:55.866555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9384dd9cea9412f91b34b5a7f690fb7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:55.867331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9384dd9cea9412f91b34b5a7f690fb7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:55.875664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9384dd9cea9412f91b34b5a7f690fb7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:55.883010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9384dd9cea9412f91b34b5a7f690fb7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:55.883702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9384dd9cea9412f91b34b5a7f690fb7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:55.903891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1205/2396] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:35:55 2022] DELETE /identity/v3/users/d9384dd9cea9412f91b34b5a7f690fb7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:55.908043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eb793-0bdc-426b-9693-59d094f9fe08 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:55.909363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eb793-0bdc-426b-9693-59d094f9fe08 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:55.909704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eb793-0bdc-426b-9693-59d094f9fe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:55.929891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-691eb793-0bdc-426b-9693-59d094f9fe08 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:55.931227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-691eb793-0bdc-426b-9693-59d094f9fe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:55.996420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1192/2397] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:35:56.005867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:56.007428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:56.082493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd1f9c-371f-4943-b4f7-60e2d5552090 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:56.086349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abed10-bab1-4adb-9ea7-86242ae2fe76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:56.087088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abed10-bab1-4adb-9ea7-86242ae2fe76 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.087415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abed10-bab1-4adb-9ea7-86242ae2fe76 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.087690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abed10-bab1-4adb-9ea7-86242ae2fe76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.097185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abed10-bab1-4adb-9ea7-86242ae2fe76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:56.098642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abed10-bab1-4adb-9ea7-86242ae2fe76 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:56.108346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1206/2398] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:35:56.384407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eb793-0bdc-426b-9693-59d094f9fe08 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:56.386945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-691eb793-0bdc-426b-9693-59d094f9fe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:56.400678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eb793-0bdc-426b-9693-59d094f9fe08 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:56.406221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:56.406983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.407268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.407536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] PATH_INFO: `/v3/projects/b5405e58b6da40edb1456103cac516d3` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.409035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5405e58b6da40edb1456103cac516d3)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:56.411106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:56.413130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:56.466903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5405e58b6da40edb1456103cac516d3', 'request_id': 'req-ef0e6e3b-cede-4c7b-86f7-105754403609'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:56.473318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5405e58b6da40edb1456103cac516d3', 'request_id': 'req-ef0e6e3b-cede-4c7b-86f7-105754403609'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:56.479888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5405e58b6da40edb1456103cac516d3', 'request_id': 'req-ef0e6e3b-cede-4c7b-86f7-105754403609'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:56.494059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] The token cache is being invalidate because project b5405e58b6da40edb1456103cac516d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:35:56.494345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:56.495022 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ef0e6e3b-cede-4c7b-86f7-105754403609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:56.496527 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1193/2399] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:35:56 2022] DELETE /identity/v3/projects/b5405e58b6da40edb1456103cac516d3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:35:56.501335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d86b-d368-495b-9268-29ab7259b68f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.501826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d86b-d368-495b-9268-29ab7259b68f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.501920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0d86b-d368-495b-9268-29ab7259b68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.522278 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-fab0d86b-d368-495b-9268-29ab7259b68f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:56.523396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fab0d86b-d368-495b-9268-29ab7259b68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:56.542908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:56.589745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1207/2400] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:56.655642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364867d-c9d9-4019-8ea4-76e28c797217 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.655918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364867d-c9d9-4019-8ea4-76e28c797217 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.656184 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364867d-c9d9-4019-8ea4-76e28c797217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.676354 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2364867d-c9d9-4019-8ea4-76e28c797217 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:56.677578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2364867d-c9d9-4019-8ea4-76e28c797217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:56.685287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1194/2401] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:56.690198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d86b-d368-495b-9268-29ab7259b68f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:56.700038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0d86b-d368-495b-9268-29ab7259b68f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:56.704856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf96dad-b824-4cd2-8433-3771ddcdbf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:56.705564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96dad-b824-4cd2-8433-3771ddcdbf6a admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.705843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96dad-b824-4cd2-8433-3771ddcdbf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.706117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf96dad-b824-4cd2-8433-3771ddcdbf6a admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.706762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf96dad-b824-4cd2-8433-3771ddcdbf6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:35:56.707066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf96dad-b824-4cd2-8433-3771ddcdbf6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:56.708702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf96dad-b824-4cd2-8433-3771ddcdbf6a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:56.720218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1208/2402] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:35:56.724667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364867d-c9d9-4019-8ea4-76e28c797217 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:56.733756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364867d-c9d9-4019-8ea4-76e28c797217 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:56.739063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fa55d-edb9-4a61-ad4b-314d7990599e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:56.739643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fa55d-edb9-4a61-ad4b-314d7990599e admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.739915 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fa55d-edb9-4a61-ad4b-314d7990599e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.740178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fa55d-edb9-4a61-ad4b-314d7990599e admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.740838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fa55d-edb9-4a61-ad4b-314d7990599e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:56.742713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fa55d-edb9-4a61-ad4b-314d7990599e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:56.769694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1195/2403] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:35:56 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:35:56.773946 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf96dad-b824-4cd2-8433-3771ddcdbf6a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:56.783933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf96dad-b824-4cd2-8433-3771ddcdbf6a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:56.840592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a1467-54e0-48af-a11c-5302951991e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:56.841195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a1467-54e0-48af-a11c-5302951991e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.841474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a1467-54e0-48af-a11c-5302951991e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.841746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a1467-54e0-48af-a11c-5302951991e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.842391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a1467-54e0-48af-a11c-5302951991e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:56.844319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a1467-54e0-48af-a11c-5302951991e5 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:56.866935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1209/2404] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:35:56 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:35:56.872373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fa55d-edb9-4a61-ad4b-314d7990599e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:56.882969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fa55d-edb9-4a61-ad4b-314d7990599e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:56.887594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a2a69-4651-4548-b463-548e67cd86fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:56.888182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a2a69-4651-4548-b463-548e67cd86fa admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.888457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a2a69-4651-4548-b463-548e67cd86fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.888759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a2a69-4651-4548-b463-548e67cd86fa admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.889429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a2a69-4651-4548-b463-548e67cd86fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:56.891021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a2a69-4651-4548-b463-548e67cd86fa admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:56.901008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1196/2405] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:35:56.904960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a1467-54e0-48af-a11c-5302951991e5 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:56.915413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a1467-54e0-48af-a11c-5302951991e5 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:56.920021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae774e-fc9a-42f5-b0c4-dd44e6563698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:56.920646 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae774e-fc9a-42f5-b0c4-dd44e6563698 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.920926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae774e-fc9a-42f5-b0c4-dd44e6563698 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.921195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae774e-fc9a-42f5-b0c4-dd44e6563698 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.921830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae774e-fc9a-42f5-b0c4-dd44e6563698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:56.923449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae774e-fc9a-42f5-b0c4-dd44e6563698 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:56.935029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1210/2406] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:35:56.939673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a2a69-4651-4548-b463-548e67cd86fa admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:56.949780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a2a69-4651-4548-b463-548e67cd86fa admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:56.954754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f6dee-fbed-47fc-89ff-b09dae84cb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:56.955400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f6dee-fbed-47fc-89ff-b09dae84cb98 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.955675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f6dee-fbed-47fc-89ff-b09dae84cb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:56.955943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f6dee-fbed-47fc-89ff-b09dae84cb98 admin admin] PATH_INFO: `/v3/projects/a5bbbbdf90054295a8bf275fbf9a164f/users/c2aec2314a0441d9b9b16a2ec680b0b7/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:56.971464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f6dee-fbed-47fc-89ff-b09dae84cb98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bbbbdf90054295a8bf275fbf9a164f, user_id=c2aec2314a0441d9b9b16a2ec680b0b7, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:56.975690 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f6dee-fbed-47fc-89ff-b09dae84cb98 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:56.988327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1197/2407] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:35:56 2022] PUT /identity/v3/projects/a5bbbbdf90054295a8bf275fbf9a164f/users/c2aec2314a0441d9b9b16a2ec680b0b7/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:35:56.999532 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f77f7-fca3-499d-bb3b-b42cffa34980 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:56.999811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f77f7-fca3-499d-bb3b-b42cffa34980 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:57.000079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f77f7-fca3-499d-bb3b-b42cffa34980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:57.030863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bc9f77f7-fca3-499d-bb3b-b42cffa34980 None None] MFA Rules not processed for user `c2aec2314a0441d9b9b16a2ec680b0b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:57.031999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bc9f77f7-fca3-499d-bb3b-b42cffa34980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:57.103173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1211/2408] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:57.115859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f6dee-fbed-47fc-89ff-b09dae84cb98 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:57.117832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d26f6dee-fbed-47fc-89ff-b09dae84cb98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:57.179355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f6dee-fbed-47fc-89ff-b09dae84cb98 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:57.185713 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee7a2a-cbb8-4452-a52e-c9efbb74c0d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52470b620abd4cd29ab7e4752f285bb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:57.186487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee7a2a-cbb8-4452-a52e-c9efbb74c0d7 service glance] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:57.186790 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee7a2a-cbb8-4452-a52e-c9efbb74c0d7 service glance] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:57.187024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee7a2a-cbb8-4452-a52e-c9efbb74c0d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:57.196367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee7a2a-cbb8-4452-a52e-c9efbb74c0d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:57.197864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee7a2a-cbb8-4452-a52e-c9efbb74c0d7 service glance] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:57.208158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1198/2409] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:35:57.926547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024e296-679a-4508-b6b8-6c9bb07f5611 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:57.927040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024e296-679a-4508-b6b8-6c9bb07f5611 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:57.927140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024e296-679a-4508-b6b8-6c9bb07f5611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:57.950252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5024e296-679a-4508-b6b8-6c9bb07f5611 None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:57.951717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5024e296-679a-4508-b6b8-6c9bb07f5611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:57.966212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:58.020195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1212/2410] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 00:35:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:35:58.032081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee7a2a-cbb8-4452-a52e-c9efbb74c0d7 service glance] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:58.033610 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-41ee7a2a-cbb8-4452-a52e-c9efbb74c0d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:58.071033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:35:58.113676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee7a2a-cbb8-4452-a52e-c9efbb74c0d7 service glance] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:58.118732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6ab73-2c84-4cb3-b884-368475e98b35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:58.119321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ab73-2c84-4cb3-b884-368475e98b35 service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:58.119599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ab73-2c84-4cb3-b884-368475e98b35 service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:58.119872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ab73-2c84-4cb3-b884-368475e98b35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:58.127914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6ab73-2c84-4cb3-b884-368475e98b35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:58.129578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6ab73-2c84-4cb3-b884-368475e98b35 service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:58.138520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1199/2411] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:35:59.703889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024e296-679a-4508-b6b8-6c9bb07f5611 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:59.704369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5024e296-679a-4508-b6b8-6c9bb07f5611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:59.780132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024e296-679a-4508-b6b8-6c9bb07f5611 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:35:59.784600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:35:59.785372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:59.785659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:59.785915 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] PATH_INFO: `/v3/users/0a6e3415ba5d491c955a9d4e44c99e4b` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:59.787110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a6e3415ba5d491c955a9d4e44c99e4b)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:35:59.789128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:35:59.838605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a6e3415ba5d491c955a9d4e44c99e4b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:59.839412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a6e3415ba5d491c955a9d4e44c99e4b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:59.847033 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a6e3415ba5d491c955a9d4e44c99e4b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:59.854481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a6e3415ba5d491c955a9d4e44c99e4b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:59.855110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a6e3415ba5d491c955a9d4e44c99e4b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:35:59.873389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1213/2412] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:35:59 2022] DELETE /identity/v3/users/0a6e3415ba5d491c955a9d4e44c99e4b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:35:59.878105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7f898-b81a-4b2b-88d9-78a81d93ef5c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:35:59.879169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7f898-b81a-4b2b-88d9-78a81d93ef5c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:35:59.879357 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7f898-b81a-4b2b-88d9-78a81d93ef5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:35:59.899024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fef7f898-b81a-4b2b-88d9-78a81d93ef5c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:35:59.900210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fef7f898-b81a-4b2b-88d9-78a81d93ef5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:35:59.961103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1200/2413] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:35:59.970719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:35:59.973037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:00.021204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:00.036880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54940c-bda3-4256-8b3c-9f4b66afb3af admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:00.041222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627412c0-cf57-4a3f-8521-b1a5ae9db8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:00.041861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627412c0-cf57-4a3f-8521-b1a5ae9db8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.042141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627412c0-cf57-4a3f-8521-b1a5ae9db8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.042412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627412c0-cf57-4a3f-8521-b1a5ae9db8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.052859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627412c0-cf57-4a3f-8521-b1a5ae9db8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:00.054301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627412c0-cf57-4a3f-8521-b1a5ae9db8d6 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:00.063564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1214/2414] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:35:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:00.352249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7f898-b81a-4b2b-88d9-78a81d93ef5c None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:00.354190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fef7f898-b81a-4b2b-88d9-78a81d93ef5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:00.368947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7f898-b81a-4b2b-88d9-78a81d93ef5c None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:00.374378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:00.375018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.375296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.375599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] PATH_INFO: `/v3/projects/fffe55914dca4883a2a77a2b017e8cde` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.376784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fffe55914dca4883a2a77a2b017e8cde)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:00.378794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:00.427837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fffe55914dca4883a2a77a2b017e8cde', 'request_id': 'req-3b63add0-4738-4385-a548-1733e5ef7db8'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:00.433460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fffe55914dca4883a2a77a2b017e8cde', 'request_id': 'req-3b63add0-4738-4385-a548-1733e5ef7db8'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:00.439849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fffe55914dca4883a2a77a2b017e8cde', 'request_id': 'req-3b63add0-4738-4385-a548-1733e5ef7db8'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:00.454314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] The token cache is being invalidate because project fffe55914dca4883a2a77a2b017e8cde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:00.454630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:00.455280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3b63add0-4738-4385-a548-1733e5ef7db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:00.457410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1201/2415] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:00 2022] DELETE /identity/v3/projects/fffe55914dca4883a2a77a2b017e8cde => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:00.462491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67225ff4-3086-430b-9060-e2a040733a8d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.462767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67225ff4-3086-430b-9060-e2a040733a8d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.463040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67225ff4-3086-430b-9060-e2a040733a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.482473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-67225ff4-3086-430b-9060-e2a040733a8d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:00.483664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-67225ff4-3086-430b-9060-e2a040733a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:00.548792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1215/2416] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:00.556263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26307f6d-10f1-4675-82d1-7de76c0a1c15 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.556577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26307f6d-10f1-4675-82d1-7de76c0a1c15 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.556849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26307f6d-10f1-4675-82d1-7de76c0a1c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.577004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-26307f6d-10f1-4675-82d1-7de76c0a1c15 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:00.578183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-26307f6d-10f1-4675-82d1-7de76c0a1c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:00.586983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1202/2417] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:00.591790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67225ff4-3086-430b-9060-e2a040733a8d None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:00.605508 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67225ff4-3086-430b-9060-e2a040733a8d None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:00.610202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca0e4a-5eb6-41d8-a260-a2488d604bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:00.610852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca0e4a-5eb6-41d8-a260-a2488d604bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.611136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca0e4a-5eb6-41d8-a260-a2488d604bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.611447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca0e4a-5eb6-41d8-a260-a2488d604bb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.612105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca0e4a-5eb6-41d8-a260-a2488d604bb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:36:00.612409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca0e4a-5eb6-41d8-a260-a2488d604bb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:00.615431 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca0e4a-5eb6-41d8-a260-a2488d604bb4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:00.627812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1216/2418] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:36:00.632311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26307f6d-10f1-4675-82d1-7de76c0a1c15 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:00.641435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26307f6d-10f1-4675-82d1-7de76c0a1c15 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:00.647096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d791bd-b1de-4ada-94bc-d3846fddb562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:00.647780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d791bd-b1de-4ada-94bc-d3846fddb562 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.648083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d791bd-b1de-4ada-94bc-d3846fddb562 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.648353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d791bd-b1de-4ada-94bc-d3846fddb562 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.649093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d791bd-b1de-4ada-94bc-d3846fddb562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:00.651198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d791bd-b1de-4ada-94bc-d3846fddb562 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:00.681530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1203/2419] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:00 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:00.685631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca0e4a-5eb6-41d8-a260-a2488d604bb4 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:00.695660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca0e4a-5eb6-41d8-a260-a2488d604bb4 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:00.754670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383e7e1-5f23-4b91-bc9e-500c1c9fe124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:00.755376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383e7e1-5f23-4b91-bc9e-500c1c9fe124 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.755555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383e7e1-5f23-4b91-bc9e-500c1c9fe124 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.755828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383e7e1-5f23-4b91-bc9e-500c1c9fe124 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.756461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383e7e1-5f23-4b91-bc9e-500c1c9fe124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:00.758463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383e7e1-5f23-4b91-bc9e-500c1c9fe124 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:00.779209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1217/2420] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:00 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:00.783001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d791bd-b1de-4ada-94bc-d3846fddb562 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:00.796256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d791bd-b1de-4ada-94bc-d3846fddb562 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:00.801852 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc57ff2-0398-433e-8c4b-77625f3024f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:00.802393 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc57ff2-0398-433e-8c4b-77625f3024f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.802668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc57ff2-0398-433e-8c4b-77625f3024f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.802933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc57ff2-0398-433e-8c4b-77625f3024f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.803578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc57ff2-0398-433e-8c4b-77625f3024f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:00.805217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc57ff2-0398-433e-8c4b-77625f3024f4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:00.817664 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1204/2421] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:00.826042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383e7e1-5f23-4b91-bc9e-500c1c9fe124 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:00.841509 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383e7e1-5f23-4b91-bc9e-500c1c9fe124 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:00.847958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f222af6-69e4-4325-bc06-91a0266d5e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:00.848675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f222af6-69e4-4325-bc06-91a0266d5e91 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.848963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f222af6-69e4-4325-bc06-91a0266d5e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.849266 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f222af6-69e4-4325-bc06-91a0266d5e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.849888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f222af6-69e4-4325-bc06-91a0266d5e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:00.851503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f222af6-69e4-4325-bc06-91a0266d5e91 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:00.863298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1218/2422] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:00.867365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc57ff2-0398-433e-8c4b-77625f3024f4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:00.873540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:00.879925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc57ff2-0398-433e-8c4b-77625f3024f4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:00.886978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a1123-711e-49d3-8949-40f40b9a12c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:00.887650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a1123-711e-49d3-8949-40f40b9a12c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.887926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a1123-711e-49d3-8949-40f40b9a12c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.888198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a1123-711e-49d3-8949-40f40b9a12c8 admin admin] PATH_INFO: `/v3/projects/888f7e6604a24b0485f3f10fac3e300f/users/eac5242e53f140088b8db205399de7c8/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.903863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a1123-711e-49d3-8949-40f40b9a12c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=888f7e6604a24b0485f3f10fac3e300f, user_id=eac5242e53f140088b8db205399de7c8, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:00.906564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a1123-711e-49d3-8949-40f40b9a12c8 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:00.919671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1205/2423] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:00 2022] PUT /identity/v3/projects/888f7e6604a24b0485f3f10fac3e300f/users/eac5242e53f140088b8db205399de7c8/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:00.930528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72131fa0-362e-4317-8b54-067fe1b35886 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:00.930848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72131fa0-362e-4317-8b54-067fe1b35886 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:00.931229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72131fa0-362e-4317-8b54-067fe1b35886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:00.962212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-72131fa0-362e-4317-8b54-067fe1b35886 None None] MFA Rules not processed for user `eac5242e53f140088b8db205399de7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:00.963425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-72131fa0-362e-4317-8b54-067fe1b35886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:01.006263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:01.032844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1219/2424] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:01.044339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a1123-711e-49d3-8949-40f40b9a12c8 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:01.046109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-cf4a1123-711e-49d3-8949-40f40b9a12c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:01.127920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a1123-711e-49d3-8949-40f40b9a12c8 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:01.134549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0f8da-6239-4812-a750-c68a88ad3ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:01.135161 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f8da-6239-4812-a750-c68a88ad3ce6 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:01.135432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f8da-6239-4812-a750-c68a88ad3ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:01.135701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0f8da-6239-4812-a750-c68a88ad3ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:01.144659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f8da-6239-4812-a750-c68a88ad3ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:01.146973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0f8da-6239-4812-a750-c68a88ad3ce6 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:01.156005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1206/2425] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:01.384957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72131fa0-362e-4317-8b54-067fe1b35886 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:01.464340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72131fa0-362e-4317-8b54-067fe1b35886 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:01.469391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:01.470229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:01.470511 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:01.471162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] PATH_INFO: `/v3/users/eac5242e53f140088b8db205399de7c8` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:01.472932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac5242e53f140088b8db205399de7c8)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:01.475514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:01.529182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac5242e53f140088b8db205399de7c8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:01.529971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac5242e53f140088b8db205399de7c8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:01.542455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac5242e53f140088b8db205399de7c8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:01.550034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac5242e53f140088b8db205399de7c8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:01.550874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac5242e53f140088b8db205399de7c8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:01.569926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1220/2426] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:01 2022] DELETE /identity/v3/users/eac5242e53f140088b8db205399de7c8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:01.574269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905fad6-e800-4377-ba21-0fdced722653 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:01.574585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905fad6-e800-4377-ba21-0fdced722653 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:01.574856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905fad6-e800-4377-ba21-0fdced722653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:01.598129 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0905fad6-e800-4377-ba21-0fdced722653 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:01.599234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0905fad6-e800-4377-ba21-0fdced722653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:01.664823 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1207/2427] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:01.674892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:01.676823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:01.741711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76e5a1-51bf-4a6e-8bcc-b7dfe4f7bc14 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:01.745409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f421fd-6ef1-4945-b55c-2909c3411772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:01.746015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f421fd-6ef1-4945-b55c-2909c3411772 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:01.746291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f421fd-6ef1-4945-b55c-2909c3411772 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:01.746559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f421fd-6ef1-4945-b55c-2909c3411772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:01.755572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f421fd-6ef1-4945-b55c-2909c3411772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:01.757043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f421fd-6ef1-4945-b55c-2909c3411772 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:01.766442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1221/2428] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:36:02.045889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905fad6-e800-4377-ba21-0fdced722653 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:02.047495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0905fad6-e800-4377-ba21-0fdced722653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:02.059794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905fad6-e800-4377-ba21-0fdced722653 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:02.064436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:02.065106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.065385 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.065652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] PATH_INFO: `/v3/projects/888f7e6604a24b0485f3f10fac3e300f` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.067362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=888f7e6604a24b0485f3f10fac3e300f)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:02.069316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:02.117258 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '888f7e6604a24b0485f3f10fac3e300f', 'request_id': 'req-18d4f893-3827-429c-9e58-38cb001896cd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:02.122624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '888f7e6604a24b0485f3f10fac3e300f', 'request_id': 'req-18d4f893-3827-429c-9e58-38cb001896cd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:02.123714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:02.132408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '888f7e6604a24b0485f3f10fac3e300f', 'request_id': 'req-18d4f893-3827-429c-9e58-38cb001896cd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:02.145430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] The token cache is being invalidate because project 888f7e6604a24b0485f3f10fac3e300f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:02.145719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:02.146339 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-18d4f893-3827-429c-9e58-38cb001896cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:02.147819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1208/2429] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:02 2022] DELETE /identity/v3/projects/888f7e6604a24b0485f3f10fac3e300f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:02.152820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13809b65-73f0-4094-881d-9929a91e43cb None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.153108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13809b65-73f0-4094-881d-9929a91e43cb None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.153384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13809b65-73f0-4094-881d-9929a91e43cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.162430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:02.299600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-13809b65-73f0-4094-881d-9929a91e43cb None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:02.302100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-13809b65-73f0-4094-881d-9929a91e43cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:02.363371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1222/2430] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:02.372729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6a1c5-1e02-4bb2-9a94-16ef594e1494 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.373068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6a1c5-1e02-4bb2-9a94-16ef594e1494 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.373346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6a1c5-1e02-4bb2-9a94-16ef594e1494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.393038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-56d6a1c5-1e02-4bb2-9a94-16ef594e1494 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:02.394941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-56d6a1c5-1e02-4bb2-9a94-16ef594e1494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:02.402877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1209/2431] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:36:02.407132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13809b65-73f0-4094-881d-9929a91e43cb None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:02.417617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13809b65-73f0-4094-881d-9929a91e43cb None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:02.422610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f6a5a-e64d-4d39-8b21-094a012c4b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:02.423548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f6a5a-e64d-4d39-8b21-094a012c4b0f admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.424371 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f6a5a-e64d-4d39-8b21-094a012c4b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.424875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f6a5a-e64d-4d39-8b21-094a012c4b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.426051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f6a5a-e64d-4d39-8b21-094a012c4b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:36:02.426594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f6a5a-e64d-4d39-8b21-094a012c4b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:02.429331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f6a5a-e64d-4d39-8b21-094a012c4b0f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:02.441457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1223/2432] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:36:02.445951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6a1c5-1e02-4bb2-9a94-16ef594e1494 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:02.456027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6a1c5-1e02-4bb2-9a94-16ef594e1494 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:02.461505 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b78726-3def-4441-9449-21c91ec886bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:02.462124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b78726-3def-4441-9449-21c91ec886bd admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.462403 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b78726-3def-4441-9449-21c91ec886bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.462677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b78726-3def-4441-9449-21c91ec886bd admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.463326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b78726-3def-4441-9449-21c91ec886bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:02.465306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b78726-3def-4441-9449-21c91ec886bd admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:02.496564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1210/2433] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:02 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:02.498891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f6a5a-e64d-4d39-8b21-094a012c4b0f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:02.509173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f6a5a-e64d-4d39-8b21-094a012c4b0f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:02.567634 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f05e5e-e781-488e-ab91-3ce09675ba55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:02.568226 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f05e5e-e781-488e-ab91-3ce09675ba55 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.568521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f05e5e-e781-488e-ab91-3ce09675ba55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.568803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f05e5e-e781-488e-ab91-3ce09675ba55 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.569414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f05e5e-e781-488e-ab91-3ce09675ba55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:02.571354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f05e5e-e781-488e-ab91-3ce09675ba55 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:02.594873 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1224/2434] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:02 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:02.600037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b78726-3def-4441-9449-21c91ec886bd admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:02.612523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b78726-3def-4441-9449-21c91ec886bd admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:02.617309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e865-d1cd-4e85-8f3c-bea90ea737e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:02.617938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e865-d1cd-4e85-8f3c-bea90ea737e1 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.618235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e865-d1cd-4e85-8f3c-bea90ea737e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.618506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e865-d1cd-4e85-8f3c-bea90ea737e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.619145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34e865-d1cd-4e85-8f3c-bea90ea737e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:02.620783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34e865-d1cd-4e85-8f3c-bea90ea737e1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:02.631636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1211/2435] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:36:02.635580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f05e5e-e781-488e-ab91-3ce09675ba55 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:02.646239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f05e5e-e781-488e-ab91-3ce09675ba55 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:02.650829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0ad65-78e9-45ee-9307-9cce51ad4aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:02.651426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0ad65-78e9-45ee-9307-9cce51ad4aac admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.651716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0ad65-78e9-45ee-9307-9cce51ad4aac admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.651972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0ad65-78e9-45ee-9307-9cce51ad4aac admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.652650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0ad65-78e9-45ee-9307-9cce51ad4aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:02.654237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0ad65-78e9-45ee-9307-9cce51ad4aac admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:02.664981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1225/2436] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:02.669194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e865-d1cd-4e85-8f3c-bea90ea737e1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:02.679684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e865-d1cd-4e85-8f3c-bea90ea737e1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:02.684276 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d76af-e8b5-482c-8dca-50fb5490fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:02.684963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d76af-e8b5-482c-8dca-50fb5490fa9d admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.685246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d76af-e8b5-482c-8dca-50fb5490fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.685517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d76af-e8b5-482c-8dca-50fb5490fa9d admin admin] PATH_INFO: `/v3/projects/d7fb43b1c2504bf5adbfe947ef5858da/users/044d966822014e30a945df838ea8ce83/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.699810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d76af-e8b5-482c-8dca-50fb5490fa9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7fb43b1c2504bf5adbfe947ef5858da, user_id=044d966822014e30a945df838ea8ce83, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:02.702778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d76af-e8b5-482c-8dca-50fb5490fa9d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:02.718153 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1212/2437] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:02 2022] PUT /identity/v3/projects/d7fb43b1c2504bf5adbfe947ef5858da/users/044d966822014e30a945df838ea8ce83/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:02.729183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8265e3-13fd-48c9-802f-87c1cfec63a7 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.729474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8265e3-13fd-48c9-802f-87c1cfec63a7 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.729750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8265e3-13fd-48c9-802f-87c1cfec63a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.758714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3a8265e3-13fd-48c9-802f-87c1cfec63a7 None None] MFA Rules not processed for user `044d966822014e30a945df838ea8ce83`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:02.759957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3a8265e3-13fd-48c9-802f-87c1cfec63a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:02.822257 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:02.827056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1226/2438] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:02.832367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d76af-e8b5-482c-8dca-50fb5490fa9d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:02.896230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d76af-e8b5-482c-8dca-50fb5490fa9d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:02.901757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08337e-3374-4693-ae63-d0dcbb102e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:02.902440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08337e-3374-4693-ae63-d0dcbb102e17 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:02.902727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08337e-3374-4693-ae63-d0dcbb102e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:02.903001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08337e-3374-4693-ae63-d0dcbb102e17 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:02.903643 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08337e-3374-4693-ae63-d0dcbb102e17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:02.905970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08337e-3374-4693-ae63-d0dcbb102e17 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:02.937049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1213/2439] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:02 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:36:02.938385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8265e3-13fd-48c9-802f-87c1cfec63a7 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:02.950093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8265e3-13fd-48c9-802f-87c1cfec63a7 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:03.007788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27a878-d356-4fa4-820c-88dc7a27341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:03.008392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27a878-d356-4fa4-820c-88dc7a27341d admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:03.008698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27a878-d356-4fa4-820c-88dc7a27341d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:03.008970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27a878-d356-4fa4-820c-88dc7a27341d admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:03.009582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27a878-d356-4fa4-820c-88dc7a27341d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:03.011500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27a878-d356-4fa4-820c-88dc7a27341d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:03.034042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1227/2440] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:02 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:03.037985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08337e-3374-4693-ae63-d0dcbb102e17 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:03.048823 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08337e-3374-4693-ae63-d0dcbb102e17 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:03.053819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04fa6a-d1a4-4376-8d9d-b72554828eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:03.054422 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04fa6a-d1a4-4376-8d9d-b72554828eda admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:03.054699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04fa6a-d1a4-4376-8d9d-b72554828eda admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:03.054966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04fa6a-d1a4-4376-8d9d-b72554828eda admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:03.055645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04fa6a-d1a4-4376-8d9d-b72554828eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:03.057274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04fa6a-d1a4-4376-8d9d-b72554828eda admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:03.069544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1214/2441] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:03.073765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27a878-d356-4fa4-820c-88dc7a27341d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:03.085573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27a878-d356-4fa4-820c-88dc7a27341d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:03.091222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045c2ca-a475-46a8-89b8-09cfb4598598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:03.091911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045c2ca-a475-46a8-89b8-09cfb4598598 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:03.092193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045c2ca-a475-46a8-89b8-09cfb4598598 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:03.092467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045c2ca-a475-46a8-89b8-09cfb4598598 admin admin] PATH_INFO: `/v3/projects/a587a0b69b7d439ba5d99e5a247e0b07/users/b646831e4b88451cb42e916f6a2117b0/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:03.108993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045c2ca-a475-46a8-89b8-09cfb4598598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a587a0b69b7d439ba5d99e5a247e0b07, user_id=b646831e4b88451cb42e916f6a2117b0, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:03.111651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045c2ca-a475-46a8-89b8-09cfb4598598 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:03.125258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1228/2442] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:03 2022] PUT /identity/v3/projects/a587a0b69b7d439ba5d99e5a247e0b07/users/b646831e4b88451cb42e916f6a2117b0/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:03.130055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04fa6a-d1a4-4376-8d9d-b72554828eda admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:03.140227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04fa6a-d1a4-4376-8d9d-b72554828eda admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:03.189718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:03.197687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b184f-610b-4820-9709-68dc59a9b107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:03.198343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b184f-610b-4820-9709-68dc59a9b107 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:03.198635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b184f-610b-4820-9709-68dc59a9b107 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:03.199086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b184f-610b-4820-9709-68dc59a9b107 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:03.199885 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b184f-610b-4820-9709-68dc59a9b107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:03.202062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b184f-610b-4820-9709-68dc59a9b107 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:03.212982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1215/2443] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:03.217076 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045c2ca-a475-46a8-89b8-09cfb4598598 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:03.227575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045c2ca-a475-46a8-89b8-09cfb4598598 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:03.232355 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253d499-aa8a-469c-a814-c5fee00edbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:03.233035 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253d499-aa8a-469c-a814-c5fee00edbf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:03.233318 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253d499-aa8a-469c-a814-c5fee00edbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:03.233584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253d499-aa8a-469c-a814-c5fee00edbf1 admin admin] PATH_INFO: `/v3/projects/a587a0b69b7d439ba5d99e5a247e0b07/users/b646831e4b88451cb42e916f6a2117b0/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:03.235469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253d499-aa8a-469c-a814-c5fee00edbf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a587a0b69b7d439ba5d99e5a247e0b07, user_id=b646831e4b88451cb42e916f6a2117b0, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:03.238867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253d499-aa8a-469c-a814-c5fee00edbf1 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:03.249274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1229/2444] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:03 2022] PUT /identity/v3/projects/a587a0b69b7d439ba5d99e5a247e0b07/users/b646831e4b88451cb42e916f6a2117b0/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:03.261160 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9723d7-b86a-4428-99f5-ecefca89b7c0 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:03.261435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9723d7-b86a-4428-99f5-ecefca89b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:03.261701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9723d7-b86a-4428-99f5-ecefca89b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:03.288601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-5e9723d7-b86a-4428-99f5-ecefca89b7c0 None None] MFA Rules not processed for user `b646831e4b88451cb42e916f6a2117b0`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:03.289803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5e9723d7-b86a-4428-99f5-ecefca89b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:03.358307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1216/2445] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:03.371569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253d499-aa8a-469c-a814-c5fee00edbf1 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:03.373341 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2253d499-aa8a-469c-a814-c5fee00edbf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:03.431580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253d499-aa8a-469c-a814-c5fee00edbf1 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:03.435631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3132a-c398-4a45-bd4b-36074c691ec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:03.436234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3132a-c398-4a45-bd4b-36074c691ec3 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:03.436533 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3132a-c398-4a45-bd4b-36074c691ec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:03.436807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3132a-c398-4a45-bd4b-36074c691ec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:03.450752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3132a-c398-4a45-bd4b-36074c691ec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:03.452660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3132a-c398-4a45-bd4b-36074c691ec3 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:03.466014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1230/2446] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:03.548419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9723d7-b86a-4428-99f5-ecefca89b7c0 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:03.558454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9723d7-b86a-4428-99f5-ecefca89b7c0 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:03.562281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a12fd-8aae-4755-a2ce-67ee18ce1bf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:03.562878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a12fd-8aae-4755-a2ce-67ee18ce1bf1 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:03.563151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a12fd-8aae-4755-a2ce-67ee18ce1bf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:03.563418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a12fd-8aae-4755-a2ce-67ee18ce1bf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:03.622957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a12fd-8aae-4755-a2ce-67ee18ce1bf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:03.624786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a12fd-8aae-4755-a2ce-67ee18ce1bf1 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:03.635100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1217/2447] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:13.459996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:13.577339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3132a-c398-4a45-bd4b-36074c691ec3 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:13.630849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=65 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:13.639408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3132a-c398-4a45-bd4b-36074c691ec3 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:13.643964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:13.644620 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:13.644898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:13.645171 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] PATH_INFO: `/v3/users/044d966822014e30a945df838ea8ce83` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:13.646485 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=044d966822014e30a945df838ea8ce83)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:13.648401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:13.691770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044d966822014e30a945df838ea8ce83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:13.692995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044d966822014e30a945df838ea8ce83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:13.700580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044d966822014e30a945df838ea8ce83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:13.707462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044d966822014e30a945df838ea8ce83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:13.708207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044d966822014e30a945df838ea8ce83'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:13.725920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1231/2448] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:13 2022] DELETE /identity/v3/users/044d966822014e30a945df838ea8ce83 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:13.730537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7886a-8497-4458-ab0e-bc2f172df69a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:13.730854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7886a-8497-4458-ab0e-bc2f172df69a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:13.731135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7886a-8497-4458-ab0e-bc2f172df69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:13.750224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0aa7886a-8497-4458-ab0e-bc2f172df69a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:13.751361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0aa7886a-8497-4458-ab0e-bc2f172df69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:13.809372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1218/2449] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:13.819736 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:13.821222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:13.881778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4bb2d-7b85-4712-8737-56fd50c0b808 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:13.885971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad392ef-cf48-4a1b-a805-870afd607ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:13.886750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad392ef-cf48-4a1b-a805-870afd607ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:13.887120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad392ef-cf48-4a1b-a805-870afd607ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:13.887390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad392ef-cf48-4a1b-a805-870afd607ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:13.897096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad392ef-cf48-4a1b-a805-870afd607ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:13.898534 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad392ef-cf48-4a1b-a805-870afd607ac4 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:13.908116 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1232/2450] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:14.178877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7886a-8497-4458-ab0e-bc2f172df69a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:14.180334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0aa7886a-8497-4458-ab0e-bc2f172df69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:14.196943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7886a-8497-4458-ab0e-bc2f172df69a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:14.201820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:14.202452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:14.202727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:14.202999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] PATH_INFO: `/v3/projects/d7fb43b1c2504bf5adbfe947ef5858da` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:14.204598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7fb43b1c2504bf5adbfe947ef5858da)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:14.206512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:14.256779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7fb43b1c2504bf5adbfe947ef5858da', 'request_id': 'req-001a7d1f-3888-45c0-bfcc-0420389a4fde'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.262198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7fb43b1c2504bf5adbfe947ef5858da', 'request_id': 'req-001a7d1f-3888-45c0-bfcc-0420389a4fde'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.268452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7fb43b1c2504bf5adbfe947ef5858da', 'request_id': 'req-001a7d1f-3888-45c0-bfcc-0420389a4fde'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.281121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] The token cache is being invalidate because project d7fb43b1c2504bf5adbfe947ef5858da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:14.281407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.282038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.283530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1219/2451] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:14 2022] DELETE /identity/v3/projects/d7fb43b1c2504bf5adbfe947ef5858da => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:36:14.288286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad392ef-cf48-4a1b-a805-870afd607ac4 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:14.297879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad392ef-cf48-4a1b-a805-870afd607ac4 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:14.316263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:14.357198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:14.357818 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:14.358089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:14.358357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] PATH_INFO: `/v3/users/b646831e4b88451cb42e916f6a2117b0` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:14.359482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b646831e4b88451cb42e916f6a2117b0)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:14.362222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:14.406571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b646831e4b88451cb42e916f6a2117b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.407200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b646831e4b88451cb42e916f6a2117b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.414690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b646831e4b88451cb42e916f6a2117b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.421515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b646831e4b88451cb42e916f6a2117b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.422131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bb31a4e8-4649-4b37-8b1b-2630cbc7f081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b646831e4b88451cb42e916f6a2117b0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.438853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1233/2452] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:14 2022] DELETE /identity/v3/users/b646831e4b88451cb42e916f6a2117b0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:14.733002 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:14.733002 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:14.747378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a7d1f-3888-45c0-bfcc-0420389a4fde admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:14.808406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:14.809066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:14.809344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:14.809613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] PATH_INFO: `/v3/projects/a587a0b69b7d439ba5d99e5a247e0b07` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:14.810744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a587a0b69b7d439ba5d99e5a247e0b07)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:14.812688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:14.844314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:14.865234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a587a0b69b7d439ba5d99e5a247e0b07', 'request_id': 'req-1b7de8fb-b40d-4e51-abd5-01dea53fede5'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.871550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a587a0b69b7d439ba5d99e5a247e0b07', 'request_id': 'req-1b7de8fb-b40d-4e51-abd5-01dea53fede5'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.877822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a587a0b69b7d439ba5d99e5a247e0b07', 'request_id': 'req-1b7de8fb-b40d-4e51-abd5-01dea53fede5'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.891192 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] The token cache is being invalidate because project a587a0b69b7d439ba5d99e5a247e0b07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:14.891472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.892096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1b7de8fb-b40d-4e51-abd5-01dea53fede5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:14.893616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1220/2453] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:14 2022] DELETE /identity/v3/projects/a587a0b69b7d439ba5d99e5a247e0b07 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:14.898888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8ac6d-1130-4ff3-b71c-b2052e8a20f8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:14.899586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8ac6d-1130-4ff3-b71c-b2052e8a20f8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:14.899586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8ac6d-1130-4ff3-b71c-b2052e8a20f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:14.920459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b9b8ac6d-1130-4ff3-b71c-b2052e8a20f8 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:14.921685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b9b8ac6d-1130-4ff3-b71c-b2052e8a20f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:14.990436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1234/2454] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:14.998616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29699c1f-d74a-4c19-8c41-69ad64ef019a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:14.998904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29699c1f-d74a-4c19-8c41-69ad64ef019a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:14.999181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29699c1f-d74a-4c19-8c41-69ad64ef019a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.020700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-29699c1f-d74a-4c19-8c41-69ad64ef019a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:15.022294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-29699c1f-d74a-4c19-8c41-69ad64ef019a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:15.032327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1221/2455] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:15.036808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8ac6d-1130-4ff3-b71c-b2052e8a20f8 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.048343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8ac6d-1130-4ff3-b71c-b2052e8a20f8 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.053757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e14f63-4193-4ac3-b2d8-13efd558ec6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.054417 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e14f63-4193-4ac3-b2d8-13efd558ec6a admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.054722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e14f63-4193-4ac3-b2d8-13efd558ec6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.055052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e14f63-4193-4ac3-b2d8-13efd558ec6a admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.055762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e14f63-4193-4ac3-b2d8-13efd558ec6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:36:15.056096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e14f63-4193-4ac3-b2d8-13efd558ec6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.058028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e14f63-4193-4ac3-b2d8-13efd558ec6a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.070643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1235/2456] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:36:15.075732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29699c1f-d74a-4c19-8c41-69ad64ef019a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.086416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29699c1f-d74a-4c19-8c41-69ad64ef019a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.091309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e17da-502d-4820-9c4f-d88473924955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.091890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e17da-502d-4820-9c4f-d88473924955 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.092173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e17da-502d-4820-9c4f-d88473924955 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.092447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e17da-502d-4820-9c4f-d88473924955 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.093122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e17da-502d-4820-9c4f-d88473924955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.095049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e17da-502d-4820-9c4f-d88473924955 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.128012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1222/2457] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:15 2022] POST /identity/v3/projects => generated 357 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:15.134375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e14f63-4193-4ac3-b2d8-13efd558ec6a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.149271 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e14f63-4193-4ac3-b2d8-13efd558ec6a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.177937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:15.216614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f6d7c-ecca-4d06-b69d-6bd5901b1ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.217218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f6d7c-ecca-4d06-b69d-6bd5901b1ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.217493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f6d7c-ecca-4d06-b69d-6bd5901b1ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.217760 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f6d7c-ecca-4d06-b69d-6bd5901b1ea7 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.218381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f6d7c-ecca-4d06-b69d-6bd5901b1ea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.220347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f6d7c-ecca-4d06-b69d-6bd5901b1ea7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.247701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1236/2458] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:15 2022] POST /identity/v3/users => generated 279 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:15.252147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e17da-502d-4820-9c4f-d88473924955 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.262769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e17da-502d-4820-9c4f-d88473924955 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.267421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f62da8-2a80-4720-b450-3feac22603de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.268021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62da8-2a80-4720-b450-3feac22603de admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.268294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62da8-2a80-4720-b450-3feac22603de admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.268598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f62da8-2a80-4720-b450-3feac22603de admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.269271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62da8-2a80-4720-b450-3feac22603de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.271442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f62da8-2a80-4720-b450-3feac22603de admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.281553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1223/2459] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:15.285686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f6d7c-ecca-4d06-b69d-6bd5901b1ea7 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.297411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f6d7c-ecca-4d06-b69d-6bd5901b1ea7 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.302122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae7648-d9df-4717-b150-1125668c0fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.302713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7648-d9df-4717-b150-1125668c0fab admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.302985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7648-d9df-4717-b150-1125668c0fab admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.303251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7648-d9df-4717-b150-1125668c0fab admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.303885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae7648-d9df-4717-b150-1125668c0fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.305635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae7648-d9df-4717-b150-1125668c0fab admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.320073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1237/2460] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:15.324320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f62da8-2a80-4720-b450-3feac22603de admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.336446 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f62da8-2a80-4720-b450-3feac22603de admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.343451 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032150-15ed-4367-8e05-3d9ef204fc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.344148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032150-15ed-4367-8e05-3d9ef204fc7a admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.344427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032150-15ed-4367-8e05-3d9ef204fc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.344770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032150-15ed-4367-8e05-3d9ef204fc7a admin admin] PATH_INFO: `/v3/projects/5b2d4f3c099841159cf71ffbd347f762/users/d5c9d736337d438290c326c227d9cc53/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.360372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41032150-15ed-4367-8e05-3d9ef204fc7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b2d4f3c099841159cf71ffbd347f762, user_id=d5c9d736337d438290c326c227d9cc53, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.363311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41032150-15ed-4367-8e05-3d9ef204fc7a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.378318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1224/2461] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:15 2022] PUT /identity/v3/projects/5b2d4f3c099841159cf71ffbd347f762/users/d5c9d736337d438290c326c227d9cc53/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:15.389069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ba2ca-af72-48da-9f89-e72383541373 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.389426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ba2ca-af72-48da-9f89-e72383541373 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.389742 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ba2ca-af72-48da-9f89-e72383541373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.421328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-da2ba2ca-af72-48da-9f89-e72383541373 None None] MFA Rules not processed for user `d5c9d736337d438290c326c227d9cc53`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:15.422551 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-da2ba2ca-af72-48da-9f89-e72383541373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:15.495146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1238/2462] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:15.499648 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032150-15ed-4367-8e05-3d9ef204fc7a admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.574751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032150-15ed-4367-8e05-3d9ef204fc7a admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.579888 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0389-1a1b-4319-a121-cf967c417b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.580525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0389-1a1b-4319-a121-cf967c417b79 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.580794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0389-1a1b-4319-a121-cf967c417b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.581059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0389-1a1b-4319-a121-cf967c417b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.581755 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f0389-1a1b-4319-a121-cf967c417b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.584848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f0389-1a1b-4319-a121-cf967c417b79 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.613532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1225/2463] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:15 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:15.617689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ba2ca-af72-48da-9f89-e72383541373 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.629084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ba2ca-af72-48da-9f89-e72383541373 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.686865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e28047-cf81-4081-9775-9cfd0706a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.687469 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e28047-cf81-4081-9775-9cfd0706a192 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.687746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e28047-cf81-4081-9775-9cfd0706a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.688014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e28047-cf81-4081-9775-9cfd0706a192 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.688693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e28047-cf81-4081-9775-9cfd0706a192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.690666 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e28047-cf81-4081-9775-9cfd0706a192 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.711945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1239/2464] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:15 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:15.716015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0389-1a1b-4319-a121-cf967c417b79 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.726876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0389-1a1b-4319-a121-cf967c417b79 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.731431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd052e9-6801-43d4-9759-ceb9f6daf68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.732028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd052e9-6801-43d4-9759-ceb9f6daf68d admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.732309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd052e9-6801-43d4-9759-ceb9f6daf68d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.732621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd052e9-6801-43d4-9759-ceb9f6daf68d admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.733374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd052e9-6801-43d4-9759-ceb9f6daf68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.734983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd052e9-6801-43d4-9759-ceb9f6daf68d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.746642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1226/2465] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:15.750750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e28047-cf81-4081-9775-9cfd0706a192 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.761646 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e28047-cf81-4081-9775-9cfd0706a192 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.766523 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e376351-880f-4b15-8de6-608a1816b041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.767281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e376351-880f-4b15-8de6-608a1816b041 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.767545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e376351-880f-4b15-8de6-608a1816b041 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.767842 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e376351-880f-4b15-8de6-608a1816b041 admin admin] PATH_INFO: `/v3/projects/f4cce483be544e9fad181f0c831207c3/users/9bd451cab6bf4ee99b25e914d8aa424e/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.783332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e376351-880f-4b15-8de6-608a1816b041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4cce483be544e9fad181f0c831207c3, user_id=9bd451cab6bf4ee99b25e914d8aa424e, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.786799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e376351-880f-4b15-8de6-608a1816b041 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.801279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1240/2466] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:15 2022] PUT /identity/v3/projects/f4cce483be544e9fad181f0c831207c3/users/9bd451cab6bf4ee99b25e914d8aa424e/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:15.805892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd052e9-6801-43d4-9759-ceb9f6daf68d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.816318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd052e9-6801-43d4-9759-ceb9f6daf68d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.837697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:15.877996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ed5ef-e2a5-4419-a52b-08e1892ea954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.878600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ed5ef-e2a5-4419-a52b-08e1892ea954 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.878881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ed5ef-e2a5-4419-a52b-08e1892ea954 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.879156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ed5ef-e2a5-4419-a52b-08e1892ea954 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.879832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ed5ef-e2a5-4419-a52b-08e1892ea954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.881526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ed5ef-e2a5-4419-a52b-08e1892ea954 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.891654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1227/2467] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:15.896786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e376351-880f-4b15-8de6-608a1816b041 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:15.902166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:15.907553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e376351-880f-4b15-8de6-608a1816b041 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:15.913732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971390da-113e-43ca-b1ad-797cafcc6889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:15.914383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971390da-113e-43ca-b1ad-797cafcc6889 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.914670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971390da-113e-43ca-b1ad-797cafcc6889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.914941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971390da-113e-43ca-b1ad-797cafcc6889 admin admin] PATH_INFO: `/v3/projects/f4cce483be544e9fad181f0c831207c3/users/9bd451cab6bf4ee99b25e914d8aa424e/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.916898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971390da-113e-43ca-b1ad-797cafcc6889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4cce483be544e9fad181f0c831207c3, user_id=9bd451cab6bf4ee99b25e914d8aa424e, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:15.919588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971390da-113e-43ca-b1ad-797cafcc6889 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:15.930273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1241/2468] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:15 2022] PUT /identity/v3/projects/f4cce483be544e9fad181f0c831207c3/users/9bd451cab6bf4ee99b25e914d8aa424e/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:15.941427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7904cc-a71d-4b61-b336-4ae59f2e46bb None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:15.941709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7904cc-a71d-4b61-b336-4ae59f2e46bb None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:15.941982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7904cc-a71d-4b61-b336-4ae59f2e46bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:15.971486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-8c7904cc-a71d-4b61-b336-4ae59f2e46bb None None] MFA Rules not processed for user `9bd451cab6bf4ee99b25e914d8aa424e`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:15.972760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8c7904cc-a71d-4b61-b336-4ae59f2e46bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:16.051124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1228/2469] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:15 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:16.062858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971390da-113e-43ca-b1ad-797cafcc6889 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:16.077136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-971390da-113e-43ca-b1ad-797cafcc6889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:16.147273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971390da-113e-43ca-b1ad-797cafcc6889 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:16.151330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236410c-b1d5-4fbb-bb2c-d221ec5b7de2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:16.151974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236410c-b1d5-4fbb-bb2c-d221ec5b7de2 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:16.152218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236410c-b1d5-4fbb-bb2c-d221ec5b7de2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:16.152487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236410c-b1d5-4fbb-bb2c-d221ec5b7de2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:16.162404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236410c-b1d5-4fbb-bb2c-d221ec5b7de2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:16.163843 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236410c-b1d5-4fbb-bb2c-d221ec5b7de2 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:16.173265 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1242/2470] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:16 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:16.247689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7904cc-a71d-4b61-b336-4ae59f2e46bb None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:16.258471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7904cc-a71d-4b61-b336-4ae59f2e46bb None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:16.262201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f997c1-0ef7-4f37-92dc-ed804b77d7b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:16.262791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f997c1-0ef7-4f37-92dc-ed804b77d7b8 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:16.263067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f997c1-0ef7-4f37-92dc-ed804b77d7b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:16.263344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f997c1-0ef7-4f37-92dc-ed804b77d7b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:16.320091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f997c1-0ef7-4f37-92dc-ed804b77d7b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:16.321557 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f997c1-0ef7-4f37-92dc-ed804b77d7b8 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:16.330606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1229/2471] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:26.122121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236410c-b1d5-4fbb-bb2c-d221ec5b7de2 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:26.123515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f236410c-b1d5-4fbb-bb2c-d221ec5b7de2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:26.185835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236410c-b1d5-4fbb-bb2c-d221ec5b7de2 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:26.190307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:26.190918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:26.191193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:26.191464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] PATH_INFO: `/v3/users/c2aec2314a0441d9b9b16a2ec680b0b7` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:26.193090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2aec2314a0441d9b9b16a2ec680b0b7)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:26.194995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:26.241262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2aec2314a0441d9b9b16a2ec680b0b7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.241940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2aec2314a0441d9b9b16a2ec680b0b7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.249333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2aec2314a0441d9b9b16a2ec680b0b7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.257110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2aec2314a0441d9b9b16a2ec680b0b7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.257771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2aec2314a0441d9b9b16a2ec680b0b7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.275321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1243/2472] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:26 2022] DELETE /identity/v3/users/c2aec2314a0441d9b9b16a2ec680b0b7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:26.279890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a988b5c-c628-44f6-922f-017581c68e43 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:26.280377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a988b5c-c628-44f6-922f-017581c68e43 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:26.280488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a988b5c-c628-44f6-922f-017581c68e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:26.298958 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3a988b5c-c628-44f6-922f-017581c68e43 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:26.300175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3a988b5c-c628-44f6-922f-017581c68e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:26.339767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:26.358032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1230/2473] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:26.368227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:26.369664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:26.383919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:26.428923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf77a99-8f7c-4cb9-916f-8e5e920ebaa0 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:26.432739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dd045-7cb0-4c87-a0e4-ed250d053b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:26.433375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dd045-7cb0-4c87-a0e4-ed250d053b5d service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:26.433680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dd045-7cb0-4c87-a0e4-ed250d053b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:26.433948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dd045-7cb0-4c87-a0e4-ed250d053b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:26.443814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dd045-7cb0-4c87-a0e4-ed250d053b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:26.445274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dd045-7cb0-4c87-a0e4-ed250d053b5d service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:26.454968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1244/2474] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:26.717996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a988b5c-c628-44f6-922f-017581c68e43 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:26.719487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3a988b5c-c628-44f6-922f-017581c68e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:26.738661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a988b5c-c628-44f6-922f-017581c68e43 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:26.744666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:26.745405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:26.745794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:26.746077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] PATH_INFO: `/v3/projects/a5bbbbdf90054295a8bf275fbf9a164f` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:26.748400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5bbbbdf90054295a8bf275fbf9a164f)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:26.749327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:26.802849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5bbbbdf90054295a8bf275fbf9a164f', 'request_id': 'req-74bf8a17-052a-4513-a7cb-600602a74857'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.808722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5bbbbdf90054295a8bf275fbf9a164f', 'request_id': 'req-74bf8a17-052a-4513-a7cb-600602a74857'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.814991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5bbbbdf90054295a8bf275fbf9a164f', 'request_id': 'req-74bf8a17-052a-4513-a7cb-600602a74857'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.830670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] The token cache is being invalidate because project a5bbbbdf90054295a8bf275fbf9a164f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:26.830967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.831851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-74bf8a17-052a-4513-a7cb-600602a74857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:26.834301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1231/2475] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:26 2022] DELETE /identity/v3/projects/a5bbbbdf90054295a8bf275fbf9a164f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:26.839313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f390c-6f4d-4f50-81a6-ad47c636419b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:26.839698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f390c-6f4d-4f50-81a6-ad47c636419b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:26.840060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f390c-6f4d-4f50-81a6-ad47c636419b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:26.859441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-9a3f390c-6f4d-4f50-81a6-ad47c636419b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:26.860767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9a3f390c-6f4d-4f50-81a6-ad47c636419b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:26.923747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1245/2476] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:26.933140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf981f4-c49c-408c-ad44-8b5eb30ecc77 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:26.933435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf981f4-c49c-408c-ad44-8b5eb30ecc77 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:26.933701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf981f4-c49c-408c-ad44-8b5eb30ecc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:26.955483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fcf981f4-c49c-408c-ad44-8b5eb30ecc77 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:26.956703 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fcf981f4-c49c-408c-ad44-8b5eb30ecc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:26.969134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1232/2477] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:26.971251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f390c-6f4d-4f50-81a6-ad47c636419b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:26.984048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f390c-6f4d-4f50-81a6-ad47c636419b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:26.988968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8724e-1c79-45c1-81dd-05e2bd1cc4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:26.989781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8724e-1c79-45c1-81dd-05e2bd1cc4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:26.992732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8724e-1c79-45c1-81dd-05e2bd1cc4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:26.993145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b8724e-1c79-45c1-81dd-05e2bd1cc4a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:26.994126 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8724e-1c79-45c1-81dd-05e2bd1cc4a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:36:26.994503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8724e-1c79-45c1-81dd-05e2bd1cc4a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:26.996214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b8724e-1c79-45c1-81dd-05e2bd1cc4a2 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:27.012564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1246/2478] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:36:27.015925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf981f4-c49c-408c-ad44-8b5eb30ecc77 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:27.026199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf981f4-c49c-408c-ad44-8b5eb30ecc77 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:27.031211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb07df-5355-4f74-9a28-1f979eafc48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:27.031935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb07df-5355-4f74-9a28-1f979eafc48e admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:27.032467 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb07df-5355-4f74-9a28-1f979eafc48e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:27.032696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb07df-5355-4f74-9a28-1f979eafc48e admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:27.033459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb07df-5355-4f74-9a28-1f979eafc48e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:27.035790 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb07df-5355-4f74-9a28-1f979eafc48e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:27.066461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1233/2479] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:27 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:27.071329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8724e-1c79-45c1-81dd-05e2bd1cc4a2 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:27.082432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b8724e-1c79-45c1-81dd-05e2bd1cc4a2 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:27.138546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb008f77-1af1-4dd1-bf86-58766cdb9da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:27.139211 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb008f77-1af1-4dd1-bf86-58766cdb9da6 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:27.139408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb008f77-1af1-4dd1-bf86-58766cdb9da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:27.139681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb008f77-1af1-4dd1-bf86-58766cdb9da6 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:27.140313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb008f77-1af1-4dd1-bf86-58766cdb9da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:27.142274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb008f77-1af1-4dd1-bf86-58766cdb9da6 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:27.156027 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb07df-5355-4f74-9a28-1f979eafc48e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:27.157763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-93eb07df-5355-4f74-9a28-1f979eafc48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:27.167314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1247/2480] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:27 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:27.168300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb008f77-1af1-4dd1-bf86-58766cdb9da6 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:27.179142 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb008f77-1af1-4dd1-bf86-58766cdb9da6 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:27.183909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36ba7-512b-45d8-bfaa-b80acc119c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:27.184598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36ba7-512b-45d8-bfaa-b80acc119c26 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:27.184951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36ba7-512b-45d8-bfaa-b80acc119c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:27.185290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c36ba7-512b-45d8-bfaa-b80acc119c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:27.185996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c36ba7-512b-45d8-bfaa-b80acc119c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:27.187665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c36ba7-512b-45d8-bfaa-b80acc119c26 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:27.199254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1248/2481] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:36:27.203378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36ba7-512b-45d8-bfaa-b80acc119c26 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:27.212909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c36ba7-512b-45d8-bfaa-b80acc119c26 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:27.217589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09574ded-8b14-40ba-a3b9-bb3819319fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:27.218170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09574ded-8b14-40ba-a3b9-bb3819319fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:27.218445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09574ded-8b14-40ba-a3b9-bb3819319fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:27.218714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09574ded-8b14-40ba-a3b9-bb3819319fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:27.219364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09574ded-8b14-40ba-a3b9-bb3819319fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:27.220969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09574ded-8b14-40ba-a3b9-bb3819319fa3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:27.221298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb07df-5355-4f74-9a28-1f979eafc48e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:27.225102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:27.225712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:27.225995 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:27.226269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:27.228044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:27.232829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1249/2482] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:36:27.236545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09574ded-8b14-40ba-a3b9-bb3819319fa3 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:27.246655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09574ded-8b14-40ba-a3b9-bb3819319fa3 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:27.251898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70839b-f208-4753-980d-70912a81453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:27.252573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70839b-f208-4753-980d-70912a81453e admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:27.252859 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70839b-f208-4753-980d-70912a81453e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:27.253131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70839b-f208-4753-980d-70912a81453e admin admin] PATH_INFO: `/v3/projects/6c6695e3c94640099c028ea4dcada9a2/users/1579f9e7c3b040faae7babe31b2b6c53/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:27.258386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:27.271054 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70839b-f208-4753-980d-70912a81453e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6695e3c94640099c028ea4dcada9a2, user_id=1579f9e7c3b040faae7babe31b2b6c53, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:27.273757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70839b-f208-4753-980d-70912a81453e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:27.287227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1250/2483] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:27 2022] PUT /identity/v3/projects/6c6695e3c94640099c028ea4dcada9a2/users/1579f9e7c3b040faae7babe31b2b6c53/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:36:27.292576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:27.296483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:27.297801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651de6a6-575b-470a-89db-f67109950cec None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:27.298025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:27.298177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651de6a6-575b-470a-89db-f67109950cec None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:27.298369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651de6a6-575b-470a-89db-f67109950cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:27.309798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1234/2484] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:36:27.328368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-651de6a6-575b-470a-89db-f67109950cec None None] MFA Rules not processed for user `1579f9e7c3b040faae7babe31b2b6c53`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:27.329582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-651de6a6-575b-470a-89db-f67109950cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:27.400767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1251/2485] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:27.412887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:27.414303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:27.479132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f8e22-5b8c-4dea-89b8-46da4047fe6c service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:27.531112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e21ab-efee-4b45-bc57-781b52f3205c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:27.531847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e21ab-efee-4b45-bc57-781b52f3205c service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:27.532127 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e21ab-efee-4b45-bc57-781b52f3205c service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:27.532562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e21ab-efee-4b45-bc57-781b52f3205c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:27.541136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e21ab-efee-4b45-bc57-781b52f3205c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:27.542612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e21ab-efee-4b45-bc57-781b52f3205c service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:27.551494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1235/2486] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:32.399563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651de6a6-575b-470a-89db-f67109950cec None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:32.401252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-651de6a6-575b-470a-89db-f67109950cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:32.474840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651de6a6-575b-470a-89db-f67109950cec None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:32.482056 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e21ab-efee-4b45-bc57-781b52f3205c service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:32.482841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31926c6f-b3a0-4216-8cb2-7e4a993e8fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:32.483874 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31926c6f-b3a0-4216-8cb2-7e4a993e8fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:32.484063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2b4e21ab-efee-4b45-bc57-781b52f3205c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:32.484202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31926c6f-b3a0-4216-8cb2-7e4a993e8fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:32.484415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31926c6f-b3a0-4216-8cb2-7e4a993e8fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:32.487642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-31926c6f-b3a0-4216-8cb2-7e4a993e8fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:32.563372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31926c6f-b3a0-4216-8cb2-7e4a993e8fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:32.564880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31926c6f-b3a0-4216-8cb2-7e4a993e8fe8 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:32.565288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e21ab-efee-4b45-bc57-781b52f3205c service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:32.570803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:32.571476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:32.571978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:32.572437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] PATH_INFO: `/v3/users/d5c9d736337d438290c326c227d9cc53` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:32.573845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c9d736337d438290c326c227d9cc53)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:32.575719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1252/2487] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:32.576531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:32.639034 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c9d736337d438290c326c227d9cc53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:32.639811 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c9d736337d438290c326c227d9cc53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:32.646667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:32.659308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c9d736337d438290c326c227d9cc53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:32.668042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c9d736337d438290c326c227d9cc53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:32.676826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c9d736337d438290c326c227d9cc53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:32.683867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1236/2488] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:32 2022] DELETE /identity/v3/users/d5c9d736337d438290c326c227d9cc53 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:36:32.688489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3646584-abc3-454e-94a9-41237350d2b8 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:32.688655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3646584-abc3-454e-94a9-41237350d2b8 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:32.688920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3646584-abc3-454e-94a9-41237350d2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:32.715223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a3646584-abc3-454e-94a9-41237350d2b8 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:32.716704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a3646584-abc3-454e-94a9-41237350d2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:32.738786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:32.787043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1253/2489] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:32.797562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:32.799166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:32.868238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a463a46-5753-461e-8dcb-ad9c9b019421 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:32.872380 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b595d-6134-4ef2-87a6-243edda197a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:32.873109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b595d-6134-4ef2-87a6-243edda197a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:32.873465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b595d-6134-4ef2-87a6-243edda197a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:32.873928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b595d-6134-4ef2-87a6-243edda197a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:32.892964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b595d-6134-4ef2-87a6-243edda197a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:32.894532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b595d-6134-4ef2-87a6-243edda197a9 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:32.906940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1237/2490] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:36:33.372779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3646584-abc3-454e-94a9-41237350d2b8 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:33.377018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a3646584-abc3-454e-94a9-41237350d2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:33.406790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3646584-abc3-454e-94a9-41237350d2b8 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:33.411950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:33.412741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:33.413206 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:33.413535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] PATH_INFO: `/v3/projects/5b2d4f3c099841159cf71ffbd347f762` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:33.414729 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b2d4f3c099841159cf71ffbd347f762)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:33.416767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:33.469645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b2d4f3c099841159cf71ffbd347f762', 'request_id': 'req-5ff488a2-bb9e-4f89-9404-3a76842b19c2'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.477903 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b2d4f3c099841159cf71ffbd347f762', 'request_id': 'req-5ff488a2-bb9e-4f89-9404-3a76842b19c2'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.485513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b2d4f3c099841159cf71ffbd347f762', 'request_id': 'req-5ff488a2-bb9e-4f89-9404-3a76842b19c2'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.499933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] The token cache is being invalidate because project 5b2d4f3c099841159cf71ffbd347f762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:33.500281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.500945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.502459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1254/2491] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:33 2022] DELETE /identity/v3/projects/5b2d4f3c099841159cf71ffbd347f762 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:33.506606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b595d-6134-4ef2-87a6-243edda197a9 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:33.516291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b595d-6134-4ef2-87a6-243edda197a9 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:33.605696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:33.606406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:33.606688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:33.606969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] PATH_INFO: `/v3/users/9bd451cab6bf4ee99b25e914d8aa424e` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:33.608504 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd451cab6bf4ee99b25e914d8aa424e)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:33.610476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:33.676518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd451cab6bf4ee99b25e914d8aa424e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.678465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd451cab6bf4ee99b25e914d8aa424e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.704761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd451cab6bf4ee99b25e914d8aa424e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.713453 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd451cab6bf4ee99b25e914d8aa424e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.716515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:33.726479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-2a310ad9-9cc1-46b9-9f62-86cf7a4bbbd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd451cab6bf4ee99b25e914d8aa424e'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:33.736953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1238/2492] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:33 2022] DELETE /identity/v3/users/9bd451cab6bf4ee99b25e914d8aa424e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:36:34.077114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:34.079137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:34.098871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff488a2-bb9e-4f89-9404-3a76842b19c2 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:34.167012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:34.167725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.167915 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.168184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] PATH_INFO: `/v3/projects/f4cce483be544e9fad181f0c831207c3` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:34.169306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4cce483be544e9fad181f0c831207c3)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:34.171202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:34.274990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4cce483be544e9fad181f0c831207c3', 'request_id': 'req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:34.277090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:34.285628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4cce483be544e9fad181f0c831207c3', 'request_id': 'req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:34.296287 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4cce483be544e9fad181f0c831207c3', 'request_id': 'req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:34.321899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] The token cache is being invalidate because project f4cce483be544e9fad181f0c831207c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:34.322780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:34.323825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e323f90b-b8ff-45a9-9e3c-03d3dea959f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:34.329980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1255/2493] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:34 2022] DELETE /identity/v3/projects/f4cce483be544e9fad181f0c831207c3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:34.333718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ad2d4-aaa0-4287-aa9d-569394ff42d0 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.334214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ad2d4-aaa0-4287-aa9d-569394ff42d0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.334594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ad2d4-aaa0-4287-aa9d-569394ff42d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:34.362269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-029ad2d4-aaa0-4287-aa9d-569394ff42d0 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:34.363828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-029ad2d4-aaa0-4287-aa9d-569394ff42d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:34.459002 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1239/2494] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:34.468514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52d8fe-cfd9-406e-8d82-f5bc5d34c6e0 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.469153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52d8fe-cfd9-406e-8d82-f5bc5d34c6e0 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.469153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52d8fe-cfd9-406e-8d82-f5bc5d34c6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:34.495556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-bf52d8fe-cfd9-406e-8d82-f5bc5d34c6e0 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:34.497475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bf52d8fe-cfd9-406e-8d82-f5bc5d34c6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:34.509882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1256/2495] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:34.514037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ad2d4-aaa0-4287-aa9d-569394ff42d0 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:34.528017 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ad2d4-aaa0-4287-aa9d-569394ff42d0 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:34.533672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb2391-58b6-4b9d-b9e8-7b3292bdbc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:34.534581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb2391-58b6-4b9d-b9e8-7b3292bdbc54 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.535048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb2391-58b6-4b9d-b9e8-7b3292bdbc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.535515 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb2391-58b6-4b9d-b9e8-7b3292bdbc54 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:34.536565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb2391-58b6-4b9d-b9e8-7b3292bdbc54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:36:34.537073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb2391-58b6-4b9d-b9e8-7b3292bdbc54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:34.539326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb2391-58b6-4b9d-b9e8-7b3292bdbc54 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:34.554037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1240/2496] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:36:34.558015 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52d8fe-cfd9-406e-8d82-f5bc5d34c6e0 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:34.568270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52d8fe-cfd9-406e-8d82-f5bc5d34c6e0 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:34.590028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394577f-685e-41b3-b148-b1a941ad12d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:34.590028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394577f-685e-41b3-b148-b1a941ad12d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.590028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394577f-685e-41b3-b148-b1a941ad12d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.590028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394577f-685e-41b3-b148-b1a941ad12d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:34.590501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394577f-685e-41b3-b148-b1a941ad12d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:34.590501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394577f-685e-41b3-b148-b1a941ad12d3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:34.620610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1257/2497] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:34 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:34.623128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb2391-58b6-4b9d-b9e8-7b3292bdbc54 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:34.634317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb2391-58b6-4b9d-b9e8-7b3292bdbc54 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:34.740973 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db0d2c-127a-49f5-bae3-692b94e44e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:34.748540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db0d2c-127a-49f5-bae3-692b94e44e39 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.748540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db0d2c-127a-49f5-bae3-692b94e44e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.748540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db0d2c-127a-49f5-bae3-692b94e44e39 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:34.748540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db0d2c-127a-49f5-bae3-692b94e44e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:34.748540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db0d2c-127a-49f5-bae3-692b94e44e39 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:34.780808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1241/2498] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:34 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:34.784643 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394577f-685e-41b3-b148-b1a941ad12d3 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:34.799118 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394577f-685e-41b3-b148-b1a941ad12d3 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:34.813598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76052b9-223f-4fe6-a9ab-da1791addfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:34.814539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76052b9-223f-4fe6-a9ab-da1791addfcb admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.815339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76052b9-223f-4fe6-a9ab-da1791addfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.815697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76052b9-223f-4fe6-a9ab-da1791addfcb admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:34.817009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76052b9-223f-4fe6-a9ab-da1791addfcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:34.820397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76052b9-223f-4fe6-a9ab-da1791addfcb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:34.836011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1258/2499] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:34.839595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db0d2c-127a-49f5-bae3-692b94e44e39 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:34.855261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db0d2c-127a-49f5-bae3-692b94e44e39 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:34.862088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9efc7-9f50-4c4a-9730-a03ee3e81327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:34.862830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9efc7-9f50-4c4a-9730-a03ee3e81327 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.863335 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9efc7-9f50-4c4a-9730-a03ee3e81327 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.863711 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9efc7-9f50-4c4a-9730-a03ee3e81327 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:34.866334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9efc7-9f50-4c4a-9730-a03ee3e81327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:34.868949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9efc7-9f50-4c4a-9730-a03ee3e81327 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:34.885119 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1242/2500] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:36:34.889303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76052b9-223f-4fe6-a9ab-da1791addfcb admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:34.901839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76052b9-223f-4fe6-a9ab-da1791addfcb admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:34.909103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930693b2-9446-4eec-b117-754837f44875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:34.917986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930693b2-9446-4eec-b117-754837f44875 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.918279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930693b2-9446-4eec-b117-754837f44875 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.919018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930693b2-9446-4eec-b117-754837f44875 admin admin] PATH_INFO: `/v3/projects/751019c651cc49ce9b421996ff7de470/users/f3a70701b96c472e95c6b3d3531a3508/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:34.940122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930693b2-9446-4eec-b117-754837f44875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=751019c651cc49ce9b421996ff7de470, user_id=f3a70701b96c472e95c6b3d3531a3508, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:34.943464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930693b2-9446-4eec-b117-754837f44875 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:34.961322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1259/2501] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:34 2022] PUT /identity/v3/projects/751019c651cc49ce9b421996ff7de470/users/f3a70701b96c472e95c6b3d3531a3508/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:34.973108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663f5f-f833-438b-a40f-403c70f2049c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:34.974311 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663f5f-f833-438b-a40f-403c70f2049c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:34.974686 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6663f5f-f833-438b-a40f-403c70f2049c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:35.005676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e6663f5f-f833-438b-a40f-403c70f2049c None None] MFA Rules not processed for user `f3a70701b96c472e95c6b3d3531a3508`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:35.006901 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e6663f5f-f833-438b-a40f-403c70f2049c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:35.055689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:35.076394 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1243/2502] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:35.081605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930693b2-9446-4eec-b117-754837f44875 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:35.152646 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930693b2-9446-4eec-b117-754837f44875 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:35.158102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128e088-ce59-4a89-95ba-62266e0dfa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:35.158825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128e088-ce59-4a89-95ba-62266e0dfa3e admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:35.159251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128e088-ce59-4a89-95ba-62266e0dfa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:35.159656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128e088-ce59-4a89-95ba-62266e0dfa3e admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:35.160396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128e088-ce59-4a89-95ba-62266e0dfa3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:35.163246 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f128e088-ce59-4a89-95ba-62266e0dfa3e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:35.192212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1260/2503] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:35 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:35.197462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6663f5f-f833-438b-a40f-403c70f2049c None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:35.209593 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6663f5f-f833-438b-a40f-403c70f2049c None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:35.284132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4440dc9-f10c-42d8-b517-c64e55e9437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:35.284785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4440dc9-f10c-42d8-b517-c64e55e9437b admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:35.285065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4440dc9-f10c-42d8-b517-c64e55e9437b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:35.285337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4440dc9-f10c-42d8-b517-c64e55e9437b admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:35.286359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4440dc9-f10c-42d8-b517-c64e55e9437b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:35.288448 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4440dc9-f10c-42d8-b517-c64e55e9437b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:35.319506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1244/2504] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:35 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:35.323489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128e088-ce59-4a89-95ba-62266e0dfa3e admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:35.334766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128e088-ce59-4a89-95ba-62266e0dfa3e admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:35.339369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a169e8b-febe-492b-9f48-09a724c061f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:35.339958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a169e8b-febe-492b-9f48-09a724c061f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:35.340263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a169e8b-febe-492b-9f48-09a724c061f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:35.340562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a169e8b-febe-492b-9f48-09a724c061f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:35.341206 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a169e8b-febe-492b-9f48-09a724c061f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:35.342808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a169e8b-febe-492b-9f48-09a724c061f3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:35.353982 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1261/2505] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:35.358081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4440dc9-f10c-42d8-b517-c64e55e9437b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:35.368006 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4440dc9-f10c-42d8-b517-c64e55e9437b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:35.373077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1a424-1e16-4cb5-b9d6-85435fe7f14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:35.373892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1a424-1e16-4cb5-b9d6-85435fe7f14e admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:35.374255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1a424-1e16-4cb5-b9d6-85435fe7f14e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:35.374551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1a424-1e16-4cb5-b9d6-85435fe7f14e admin admin] PATH_INFO: `/v3/projects/fcb1c9e3389f411d94b5694ca8763aa8/users/c639a73663d24aac8bfa323d189b41cb/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:35.392531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1a424-1e16-4cb5-b9d6-85435fe7f14e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb1c9e3389f411d94b5694ca8763aa8, user_id=c639a73663d24aac8bfa323d189b41cb, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:35.395887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1a424-1e16-4cb5-b9d6-85435fe7f14e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:35.409666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1245/2506] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:35 2022] PUT /identity/v3/projects/fcb1c9e3389f411d94b5694ca8763aa8/users/c639a73663d24aac8bfa323d189b41cb/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:35.414466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a169e8b-febe-492b-9f48-09a724c061f3 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:35.426456 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a169e8b-febe-492b-9f48-09a724c061f3 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:35.475621 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:35.493474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9720f1-5905-4a59-b29d-226b2f8d15c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:35.494737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9720f1-5905-4a59-b29d-226b2f8d15c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:35.495222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9720f1-5905-4a59-b29d-226b2f8d15c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:35.496840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9720f1-5905-4a59-b29d-226b2f8d15c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:35.497880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9720f1-5905-4a59-b29d-226b2f8d15c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:35.499630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9720f1-5905-4a59-b29d-226b2f8d15c4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:35.514261 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1262/2507] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:35.516983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1a424-1e16-4cb5-b9d6-85435fe7f14e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:35.529067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1a424-1e16-4cb5-b9d6-85435fe7f14e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:35.533747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83400e-ba95-48df-833a-0c6be99ac9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:35.534493 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83400e-ba95-48df-833a-0c6be99ac9af admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:35.534839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83400e-ba95-48df-833a-0c6be99ac9af admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:35.535179 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83400e-ba95-48df-833a-0c6be99ac9af admin admin] PATH_INFO: `/v3/projects/fcb1c9e3389f411d94b5694ca8763aa8/users/c639a73663d24aac8bfa323d189b41cb/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:35.537965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83400e-ba95-48df-833a-0c6be99ac9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb1c9e3389f411d94b5694ca8763aa8, user_id=c639a73663d24aac8bfa323d189b41cb, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:35.541382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83400e-ba95-48df-833a-0c6be99ac9af admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:35.556613 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1246/2508] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:35 2022] PUT /identity/v3/projects/fcb1c9e3389f411d94b5694ca8763aa8/users/c639a73663d24aac8bfa323d189b41cb/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:35.566619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8df71-e7d0-448b-8fc2-ff449a42098d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:35.566906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8df71-e7d0-448b-8fc2-ff449a42098d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:35.567253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8df71-e7d0-448b-8fc2-ff449a42098d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:35.598021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-a7f8df71-e7d0-448b-8fc2-ff449a42098d None None] MFA Rules not processed for user `c639a73663d24aac8bfa323d189b41cb`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:35.599556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a7f8df71-e7d0-448b-8fc2-ff449a42098d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:35.684305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1263/2509] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:35.698763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83400e-ba95-48df-833a-0c6be99ac9af admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:35.702203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-de83400e-ba95-48df-833a-0c6be99ac9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:35.782899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83400e-ba95-48df-833a-0c6be99ac9af admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:35.790186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdd233-7113-4a37-97cf-3236411a6253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:35.790946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdd233-7113-4a37-97cf-3236411a6253 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:35.791375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdd233-7113-4a37-97cf-3236411a6253 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:35.791650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdd233-7113-4a37-97cf-3236411a6253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:35.805248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdd233-7113-4a37-97cf-3236411a6253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:35.806734 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdd233-7113-4a37-97cf-3236411a6253 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:35.827780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1247/2510] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:35.904060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8df71-e7d0-448b-8fc2-ff449a42098d None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:35.924858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8df71-e7d0-448b-8fc2-ff449a42098d None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:35.935329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fe1fa-88df-4486-811e-bbe3a36ea130 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:35.936056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fe1fa-88df-4486-811e-bbe3a36ea130 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:35.936459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fe1fa-88df-4486-811e-bbe3a36ea130 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:35.936810 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fe1fa-88df-4486-811e-bbe3a36ea130 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:36.039343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fe1fa-88df-4486-811e-bbe3a36ea130 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:36.041429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fe1fa-88df-4486-811e-bbe3a36ea130 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:36.064392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1264/2511] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:38.366035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdd233-7113-4a37-97cf-3236411a6253 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:38.367455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0cfdd233-7113-4a37-97cf-3236411a6253 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:38.437950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:38.458612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdd233-7113-4a37-97cf-3236411a6253 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:38.463163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:38.463798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:38.464077 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:38.464341 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] PATH_INFO: `/v3/users/1579f9e7c3b040faae7babe31b2b6c53` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:38.465501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1579f9e7c3b040faae7babe31b2b6c53)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:38.467480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:38.516211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1579f9e7c3b040faae7babe31b2b6c53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:38.516903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1579f9e7c3b040faae7babe31b2b6c53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:38.531520 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1579f9e7c3b040faae7babe31b2b6c53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:38.542760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1579f9e7c3b040faae7babe31b2b6c53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:38.550791 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1579f9e7c3b040faae7babe31b2b6c53'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:38.558215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1248/2512] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:38 2022] DELETE /identity/v3/users/1579f9e7c3b040faae7babe31b2b6c53 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:38.562919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9005c5-7339-4643-a84d-96ac678d1c72 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:38.563373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9005c5-7339-4643-a84d-96ac678d1c72 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:38.563669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9005c5-7339-4643-a84d-96ac678d1c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:38.585449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5d9005c5-7339-4643-a84d-96ac678d1c72 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:38.586668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5d9005c5-7339-4643-a84d-96ac678d1c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:38.649239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1265/2513] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:38.658989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:38.660406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:38.720343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a67ce7-534e-4b2e-bfd4-66f0251cf741 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:38.725407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b29e1-5360-453f-99e3-7853041456c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:38.725407 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b29e1-5360-453f-99e3-7853041456c3 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:38.725649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b29e1-5360-453f-99e3-7853041456c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:38.726015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b29e1-5360-453f-99e3-7853041456c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:38.735358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b29e1-5360-453f-99e3-7853041456c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:38.737059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b29e1-5360-453f-99e3-7853041456c3 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:38.749565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1249/2514] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:39.260350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9005c5-7339-4643-a84d-96ac678d1c72 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:39.263675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5d9005c5-7339-4643-a84d-96ac678d1c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:39.284460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:39.297414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b29e1-5360-453f-99e3-7853041456c3 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:39.298982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a80b29e1-5360-453f-99e3-7853041456c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:39.319880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b29e1-5360-453f-99e3-7853041456c3 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:39.335517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:39.337317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:39.337854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:39.338889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] PATH_INFO: `/v3/projects/6c6695e3c94640099c028ea4dcada9a2` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:39.342807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6695e3c94640099c028ea4dcada9a2)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:39.353216 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:39.429788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9005c5-7339-4643-a84d-96ac678d1c72 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:39.452251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7586f7-6436-4b5c-9add-8d73890dcf85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:39.453288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7586f7-6436-4b5c-9add-8d73890dcf85 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:39.454603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7586f7-6436-4b5c-9add-8d73890dcf85 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:39.455042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7586f7-6436-4b5c-9add-8d73890dcf85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:39.461339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3a7586f7-6436-4b5c-9add-8d73890dcf85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:39.471381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6695e3c94640099c028ea4dcada9a2', 'request_id': 'req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:39.486714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6695e3c94640099c028ea4dcada9a2', 'request_id': 'req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:39.491817 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:39.506036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6695e3c94640099c028ea4dcada9a2', 'request_id': 'req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:39.528424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] The token cache is being invalidate because project 6c6695e3c94640099c028ea4dcada9a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:39.529200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:39.532450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac9cb853-29c0-4c1d-a7d6-af6046c73b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:39.534490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1250/2515] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:39 2022] DELETE /identity/v3/projects/6c6695e3c94640099c028ea4dcada9a2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:39.541294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46e2d6-6fc3-4981-b21c-af5f8c33713b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:39.542497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46e2d6-6fc3-4981-b21c-af5f8c33713b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:39.542804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46e2d6-6fc3-4981-b21c-af5f8c33713b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:39.568864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3f46e2d6-6fc3-4981-b21c-af5f8c33713b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:39.570273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3f46e2d6-6fc3-4981-b21c-af5f8c33713b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:39.586493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7586f7-6436-4b5c-9add-8d73890dcf85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:39.588450 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7586f7-6436-4b5c-9add-8d73890dcf85 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:39.608854 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1266/2516] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:39.653070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1251/2517] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:39.663482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dcd73-1c1d-4b11-b5f3-923267aa0746 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:39.663745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dcd73-1c1d-4b11-b5f3-923267aa0746 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:39.664020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559dcd73-1c1d-4b11-b5f3-923267aa0746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:39.701360 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-559dcd73-1c1d-4b11-b5f3-923267aa0746 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:39.703698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-559dcd73-1c1d-4b11-b5f3-923267aa0746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:39.727345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1267/2518] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:39.734251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46e2d6-6fc3-4981-b21c-af5f8c33713b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:39.757833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46e2d6-6fc3-4981-b21c-af5f8c33713b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:39.767314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:39.768235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:39.768699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:39.769220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:39.770900 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:36:39.771433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:39.774839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:39.793575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1252/2519] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:36:39.799557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559dcd73-1c1d-4b11-b5f3-923267aa0746 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:39.810548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559dcd73-1c1d-4b11-b5f3-923267aa0746 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:39.815913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34344656-ca3a-4892-b164-416c416c289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:39.817701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34344656-ca3a-4892-b164-416c416c289b admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:39.818559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34344656-ca3a-4892-b164-416c416c289b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:39.818559 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34344656-ca3a-4892-b164-416c416c289b admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:39.819587 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34344656-ca3a-4892-b164-416c416c289b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:39.823229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34344656-ca3a-4892-b164-416c416c289b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:39.875741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:39.877677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:39.887131 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1268/2520] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:39 2022] POST /identity/v3/projects => generated 391 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:39.892779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34344656-ca3a-4892-b164-416c416c289b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:39.897978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05b2fc-a908-4d46-b46d-c29bf75f3af9 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:39.982888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34344656-ca3a-4892-b164-416c416c289b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:39.993721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259564f6-cd0b-4746-9861-8dc1545f4a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:39.994415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259564f6-cd0b-4746-9861-8dc1545f4a36 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:39.994747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259564f6-cd0b-4746-9861-8dc1545f4a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:39.995052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259564f6-cd0b-4746-9861-8dc1545f4a36 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:39.995715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259564f6-cd0b-4746-9861-8dc1545f4a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:39.997847 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259564f6-cd0b-4746-9861-8dc1545f4a36 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:39.998003 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:39.998494 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:39.998773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:39.999117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] PATH_INFO: `/v3/users/af0e23f782c74c5d896ab24b64b43b18` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:40.000522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0e23f782c74c5d896ab24b64b43b18)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:40.003082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:40.023979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1269/2521] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:39 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:40.028927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259564f6-cd0b-4746-9861-8dc1545f4a36 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:40.040929 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259564f6-cd0b-4746-9861-8dc1545f4a36 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:40.050852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b15a9-e022-4894-bd33-a083bb5feda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:40.051679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b15a9-e022-4894-bd33-a083bb5feda3 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:40.064853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b15a9-e022-4894-bd33-a083bb5feda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:40.064853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b15a9-e022-4894-bd33-a083bb5feda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:40.064853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b15a9-e022-4894-bd33-a083bb5feda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:40.064853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b15a9-e022-4894-bd33-a083bb5feda3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:40.065237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0e23f782c74c5d896ab24b64b43b18'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.065335 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:40.065542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0e23f782c74c5d896ab24b64b43b18'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.082262 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1270/2522] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:40.084486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b15a9-e022-4894-bd33-a083bb5feda3 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:40.087402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0e23f782c74c5d896ab24b64b43b18'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.099886 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0e23f782c74c5d896ab24b64b43b18'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.102148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b15a9-e022-4894-bd33-a083bb5feda3 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:40.105282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:40.118823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698741e-8b06-4e4e-a08e-b8ef1e7ce4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:40.119770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698741e-8b06-4e4e-a08e-b8ef1e7ce4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:40.120464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698741e-8b06-4e4e-a08e-b8ef1e7ce4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:40.121637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0e23f782c74c5d896ab24b64b43b18'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.122221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698741e-8b06-4e4e-a08e-b8ef1e7ce4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:40.124343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698741e-8b06-4e4e-a08e-b8ef1e7ce4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:40.125980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698741e-8b06-4e4e-a08e-b8ef1e7ce4e8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:40.131237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1253/2523] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:39 2022] DELETE /identity/v3/users/af0e23f782c74c5d896ab24b64b43b18 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:36:40.144245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1271/2524] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:40.145938 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:40.147443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:40.222081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeed5c9-0c81-4722-8dda-9eab6fd85c71 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:40.227602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316b5a8-f981-499c-9cb7-0b211c607f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:40.228742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316b5a8-f981-499c-9cb7-0b211c607f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:40.228867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316b5a8-f981-499c-9cb7-0b211c607f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:40.229036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316b5a8-f981-499c-9cb7-0b211c607f7a admin admin] PATH_INFO: `/v3/projects/c035d676c6f64b7fb4d7a4039e3d86b1/users/9831a45115484d0680771badd8302ff4/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:40.246476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316b5a8-f981-499c-9cb7-0b211c607f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c035d676c6f64b7fb4d7a4039e3d86b1, user_id=9831a45115484d0680771badd8302ff4, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:40.249199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316b5a8-f981-499c-9cb7-0b211c607f7a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:40.264589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1254/2525] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:40 2022] PUT /identity/v3/projects/c035d676c6f64b7fb4d7a4039e3d86b1/users/9831a45115484d0680771badd8302ff4/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:36:40.278812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e652907-b6a0-49a3-808b-7f2e3b3e6e8f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:40.279092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e652907-b6a0-49a3-808b-7f2e3b3e6e8f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:40.279359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e652907-b6a0-49a3-808b-7f2e3b3e6e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:40.310097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3e652907-b6a0-49a3-808b-7f2e3b3e6e8f None None] MFA Rules not processed for user `9831a45115484d0680771badd8302ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:40.311636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3e652907-b6a0-49a3-808b-7f2e3b3e6e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:40.323618 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316b5a8-f981-499c-9cb7-0b211c607f7a admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:40.325836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0316b5a8-f981-499c-9cb7-0b211c607f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:40.390100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1272/2526] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:40.399785 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316b5a8-f981-499c-9cb7-0b211c607f7a admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:40.402021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e652907-b6a0-49a3-808b-7f2e3b3e6e8f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:40.403399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3e652907-b6a0-49a3-808b-7f2e3b3e6e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:40.405055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:40.405692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:40.406122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:40.406411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] PATH_INFO: `/v3/projects/3f39cdb9221440658d7cd52c341cf08a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:40.407543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f39cdb9221440658d7cd52c341cf08a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:40.409763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:40.459439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f39cdb9221440658d7cd52c341cf08a', 'request_id': 'req-ed7cbd98-02df-4460-bb1c-752755347e9c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.464797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e652907-b6a0-49a3-808b-7f2e3b3e6e8f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:40.465010 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f39cdb9221440658d7cd52c341cf08a', 'request_id': 'req-ed7cbd98-02df-4460-bb1c-752755347e9c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.471475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f39cdb9221440658d7cd52c341cf08a', 'request_id': 'req-ed7cbd98-02df-4460-bb1c-752755347e9c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.485830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] The token cache is being invalidate because project 3f39cdb9221440658d7cd52c341cf08a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:40.486099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.486775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.488332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1255/2527] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:40 2022] DELETE /identity/v3/projects/3f39cdb9221440658d7cd52c341cf08a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:40.494221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:40.496249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:40.510301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:40.518487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836749e7-6705-4458-a8e7-b57a942fadb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:40.519101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836749e7-6705-4458-a8e7-b57a942fadb4 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:40.519380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836749e7-6705-4458-a8e7-b57a942fadb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:40.519653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836749e7-6705-4458-a8e7-b57a942fadb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:40.522911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-836749e7-6705-4458-a8e7-b57a942fadb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:40.573980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:40.578210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cbd98-02df-4460-bb1c-752755347e9c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:40.583310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:40.583948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:40.584222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:40.584490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] PATH_INFO: `/v3/users/6e123c6435e746eab4d3bf8d425f3186` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:40.585642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e123c6435e746eab4d3bf8d425f3186)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:40.587561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:40.589165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836749e7-6705-4458-a8e7-b57a942fadb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:40.590635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836749e7-6705-4458-a8e7-b57a942fadb4 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:40.600145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1273/2528] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:36:40.635315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e123c6435e746eab4d3bf8d425f3186'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.635975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e123c6435e746eab4d3bf8d425f3186'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.648794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e123c6435e746eab4d3bf8d425f3186'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.655956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e123c6435e746eab4d3bf8d425f3186'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.664250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e123c6435e746eab4d3bf8d425f3186'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:40.670549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1256/2529] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:40 2022] DELETE /identity/v3/users/6e123c6435e746eab4d3bf8d425f3186 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:36:40.899363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836749e7-6705-4458-a8e7-b57a942fadb4 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:40.944633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836749e7-6705-4458-a8e7-b57a942fadb4 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:41.015981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:41.016642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:41.016972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:41.017256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] PATH_INFO: `/v3/projects/a4f6de578de64bcb8175fe1bd87b025e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:41.018357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f6de578de64bcb8175fe1bd87b025e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:41.020328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:41.065249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f6de578de64bcb8175fe1bd87b025e', 'request_id': 'req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:41.071129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f6de578de64bcb8175fe1bd87b025e', 'request_id': 'req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:41.077589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f6de578de64bcb8175fe1bd87b025e', 'request_id': 'req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:41.089782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] The token cache is being invalidate because project a4f6de578de64bcb8175fe1bd87b025e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:41.090071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:41.090686 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a4d60d6e-e3c8-4e31-a7f8-bb644a7cd11e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:41.092196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1274/2530] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:40 2022] DELETE /identity/v3/projects/a4f6de578de64bcb8175fe1bd87b025e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:50.667090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:51.086850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:51.324021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:51.325893 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:51.340261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82cf1b-21ed-4a5d-aac1-b87b378af663 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:51.400319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:51.400990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:51.401273 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:51.401548 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] PATH_INFO: `/v3/users/f3a70701b96c472e95c6b3d3531a3508` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:51.402691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a70701b96c472e95c6b3d3531a3508)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:51.404669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:51.459966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a70701b96c472e95c6b3d3531a3508'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:51.460774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a70701b96c472e95c6b3d3531a3508'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:51.472842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a70701b96c472e95c6b3d3531a3508'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:51.480757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a70701b96c472e95c6b3d3531a3508'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:51.491505 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a70701b96c472e95c6b3d3531a3508'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:51.497917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1257/2531] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:51 2022] DELETE /identity/v3/users/f3a70701b96c472e95c6b3d3531a3508 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:51.502489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95272d8-0809-4fb6-9112-cfacb73498a1 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:51.502806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95272d8-0809-4fb6-9112-cfacb73498a1 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:51.503132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95272d8-0809-4fb6-9112-cfacb73498a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:51.523761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e95272d8-0809-4fb6-9112-cfacb73498a1 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:51.524998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e95272d8-0809-4fb6-9112-cfacb73498a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:51.585802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1275/2532] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:51.595438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:51.596944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:51.682840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49a6e-8eb7-4921-8a2e-1c3fc31e68f5 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:51.686638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850f428-3cfe-423a-b0fb-906959503e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:51.687249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850f428-3cfe-423a-b0fb-906959503e8d service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:51.687525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850f428-3cfe-423a-b0fb-906959503e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:51.687797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850f428-3cfe-423a-b0fb-906959503e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:51.698425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850f428-3cfe-423a-b0fb-906959503e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:51.700396 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850f428-3cfe-423a-b0fb-906959503e8d service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:51.711112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1258/2533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:51.986525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95272d8-0809-4fb6-9112-cfacb73498a1 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:51.988082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e95272d8-0809-4fb6-9112-cfacb73498a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:52.000249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95272d8-0809-4fb6-9112-cfacb73498a1 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:52.005473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:52.006103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:52.006409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:52.006681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] PATH_INFO: `/v3/projects/751019c651cc49ce9b421996ff7de470` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:52.007957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=751019c651cc49ce9b421996ff7de470)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:52.009925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:52.057462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751019c651cc49ce9b421996ff7de470', 'request_id': 'req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.062876 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751019c651cc49ce9b421996ff7de470', 'request_id': 'req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.068887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751019c651cc49ce9b421996ff7de470', 'request_id': 'req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.081388 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] The token cache is being invalidate because project 751019c651cc49ce9b421996ff7de470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:52.081717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.082861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.084715 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1276/2534] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:51 2022] DELETE /identity/v3/projects/751019c651cc49ce9b421996ff7de470 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:52.088978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850f428-3cfe-423a-b0fb-906959503e8d service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:52.099236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850f428-3cfe-423a-b0fb-906959503e8d service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:52.117828 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:52.157752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:52.158376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:52.158647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:52.158912 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] PATH_INFO: `/v3/users/c639a73663d24aac8bfa323d189b41cb` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:52.160004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c639a73663d24aac8bfa323d189b41cb)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:52.161969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:52.207121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c639a73663d24aac8bfa323d189b41cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.208343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c639a73663d24aac8bfa323d189b41cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.220410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c639a73663d24aac8bfa323d189b41cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.227365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c639a73663d24aac8bfa323d189b41cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.236005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d13ac1f9-6d7b-4ca4-92bd-238090209e86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c639a73663d24aac8bfa323d189b41cb'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.242104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1259/2535] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:36:52 2022] DELETE /identity/v3/users/c639a73663d24aac8bfa323d189b41cb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:36:52.518570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:52.521103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:52.534250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e339-34a1-46d5-8ba7-f9dcfd630da6 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:52.592120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:52.592796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:52.593079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:52.593352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] PATH_INFO: `/v3/projects/fcb1c9e3389f411d94b5694ca8763aa8` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:52.594471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb1c9e3389f411d94b5694ca8763aa8)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:52.596429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:52.626937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:52.644728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb1c9e3389f411d94b5694ca8763aa8', 'request_id': 'req-0d669be5-a5f7-49a8-8754-cd773326dd78'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.650341 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb1c9e3389f411d94b5694ca8763aa8', 'request_id': 'req-0d669be5-a5f7-49a8-8754-cd773326dd78'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.656281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb1c9e3389f411d94b5694ca8763aa8', 'request_id': 'req-0d669be5-a5f7-49a8-8754-cd773326dd78'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.669162 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] The token cache is being invalidate because project fcb1c9e3389f411d94b5694ca8763aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:36:52.669449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.670107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-0d669be5-a5f7-49a8-8754-cd773326dd78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:36:52.671693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1277/2536] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:36:52 2022] DELETE /identity/v3/projects/fcb1c9e3389f411d94b5694ca8763aa8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:36:52.677103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027a744-7b1e-43e0-814b-7d786600d2d4 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:52.677632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027a744-7b1e-43e0-814b-7d786600d2d4 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:52.677906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027a744-7b1e-43e0-814b-7d786600d2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:52.699965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-a027a744-7b1e-43e0-814b-7d786600d2d4 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:52.701253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a027a744-7b1e-43e0-814b-7d786600d2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:52.766966 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1260/2537] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:52.775073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9234e-6c0c-4f11-8ebc-cb0dd25dc585 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:52.775364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9234e-6c0c-4f11-8ebc-cb0dd25dc585 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:52.775633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9234e-6c0c-4f11-8ebc-cb0dd25dc585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:52.795773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-72b9234e-6c0c-4f11-8ebc-cb0dd25dc585 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:52.796999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-72b9234e-6c0c-4f11-8ebc-cb0dd25dc585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:52.806632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1278/2538] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:52.811094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027a744-7b1e-43e0-814b-7d786600d2d4 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:52.821677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027a744-7b1e-43e0-814b-7d786600d2d4 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:52.826568 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac73da-0100-4db4-a037-95a60a22e2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:52.827166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac73da-0100-4db4-a037-95a60a22e2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:52.827443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac73da-0100-4db4-a037-95a60a22e2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:52.827715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac73da-0100-4db4-a037-95a60a22e2b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:52.828431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac73da-0100-4db4-a037-95a60a22e2b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:36:52.828766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac73da-0100-4db4-a037-95a60a22e2b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:52.830364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac73da-0100-4db4-a037-95a60a22e2b4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:52.841845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1261/2539] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:36:52.846177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9234e-6c0c-4f11-8ebc-cb0dd25dc585 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:52.855330 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9234e-6c0c-4f11-8ebc-cb0dd25dc585 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:52.860157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473718e-8ec4-464f-bd6a-1a32e0d0d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:52.860780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473718e-8ec4-464f-bd6a-1a32e0d0d06d admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:52.861059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473718e-8ec4-464f-bd6a-1a32e0d0d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:52.861327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473718e-8ec4-464f-bd6a-1a32e0d0d06d admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:52.861951 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473718e-8ec4-464f-bd6a-1a32e0d0d06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:52.864480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473718e-8ec4-464f-bd6a-1a32e0d0d06d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:52.895151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1279/2540] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:36:52 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:52.899577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac73da-0100-4db4-a037-95a60a22e2b4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:52.909466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac73da-0100-4db4-a037-95a60a22e2b4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:52.935962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:52.968320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335e275-4ab0-4952-9b9d-c307b67876a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:52.968962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335e275-4ab0-4952-9b9d-c307b67876a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:52.969236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335e275-4ab0-4952-9b9d-c307b67876a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:52.969500 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335e275-4ab0-4952-9b9d-c307b67876a0 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:52.970109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335e275-4ab0-4952-9b9d-c307b67876a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:52.972011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335e275-4ab0-4952-9b9d-c307b67876a0 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:52.993275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1262/2541] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:36:52 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:36:52.997516 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473718e-8ec4-464f-bd6a-1a32e0d0d06d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:53.007965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473718e-8ec4-464f-bd6a-1a32e0d0d06d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:53.012543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668cec2-86cd-4c54-b7d2-2cfecc12f016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:53.013158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668cec2-86cd-4c54-b7d2-2cfecc12f016 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:53.013420 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668cec2-86cd-4c54-b7d2-2cfecc12f016 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:53.013690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668cec2-86cd-4c54-b7d2-2cfecc12f016 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:53.014327 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668cec2-86cd-4c54-b7d2-2cfecc12f016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:53.015913 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668cec2-86cd-4c54-b7d2-2cfecc12f016 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:53.025948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1280/2542] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:36:53.029987 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335e275-4ab0-4952-9b9d-c307b67876a0 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:53.040559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335e275-4ab0-4952-9b9d-c307b67876a0 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:53.045131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ea5ee-4abc-4ffb-85e6-f54f6d503b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:53.045719 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ea5ee-4abc-4ffb-85e6-f54f6d503b63 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:53.045996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ea5ee-4abc-4ffb-85e6-f54f6d503b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:53.046263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ea5ee-4abc-4ffb-85e6-f54f6d503b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:53.046898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ea5ee-4abc-4ffb-85e6-f54f6d503b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:53.048484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ea5ee-4abc-4ffb-85e6-f54f6d503b63 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:53.058984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1263/2543] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:36:53.063172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668cec2-86cd-4c54-b7d2-2cfecc12f016 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:53.073364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668cec2-86cd-4c54-b7d2-2cfecc12f016 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:53.078180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd84cd-b1bf-4227-ae42-2c38d6426825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:53.078811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd84cd-b1bf-4227-ae42-2c38d6426825 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:53.079085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd84cd-b1bf-4227-ae42-2c38d6426825 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:53.079354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd84cd-b1bf-4227-ae42-2c38d6426825 admin admin] PATH_INFO: `/v3/projects/cfac65018f1545e08a4fdf92a158606d/users/abe2a0e2a75e45a7a1ac301e5f7d3dca/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:53.093536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd84cd-b1bf-4227-ae42-2c38d6426825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfac65018f1545e08a4fdf92a158606d, user_id=abe2a0e2a75e45a7a1ac301e5f7d3dca, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:53.096181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd84cd-b1bf-4227-ae42-2c38d6426825 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:53.109863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1281/2544] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:36:53 2022] PUT /identity/v3/projects/cfac65018f1545e08a4fdf92a158606d/users/abe2a0e2a75e45a7a1ac301e5f7d3dca/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:36:53.121706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079ef1-6830-4068-9e41-0cebb2ffba6a None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:53.122018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079ef1-6830-4068-9e41-0cebb2ffba6a None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:53.122295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079ef1-6830-4068-9e41-0cebb2ffba6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:53.149474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3b079ef1-6830-4068-9e41-0cebb2ffba6a None None] MFA Rules not processed for user `abe2a0e2a75e45a7a1ac301e5f7d3dca`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:36:53.150715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3b079ef1-6830-4068-9e41-0cebb2ffba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:53.213862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1264/2545] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:36:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:36:53.225294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd84cd-b1bf-4227-ae42-2c38d6426825 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:53.226940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3acd84cd-b1bf-4227-ae42-2c38d6426825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:53.283819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd84cd-b1bf-4227-ae42-2c38d6426825 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:53.287930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682697a-08aa-4881-ab78-897215542db4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:53.288563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682697a-08aa-4881-ab78-897215542db4 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:53.288846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682697a-08aa-4881-ab78-897215542db4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:53.289130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682697a-08aa-4881-ab78-897215542db4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:53.297368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682697a-08aa-4881-ab78-897215542db4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:53.298798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682697a-08aa-4881-ab78-897215542db4 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:53.307428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1282/2546] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:36:59.382220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079ef1-6830-4068-9e41-0cebb2ffba6a None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:36:59.383739 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3b079ef1-6830-4068-9e41-0cebb2ffba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:59.468149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079ef1-6830-4068-9e41-0cebb2ffba6a None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:36:59.472677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5ece6-90a2-48e2-b48c-52e832483d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:36:59.473299 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5ece6-90a2-48e2-b48c-52e832483d9a service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:36:59.474793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5ece6-90a2-48e2-b48c-52e832483d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:36:59.475072 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5ece6-90a2-48e2-b48c-52e832483d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:36:59.477019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5de5ece6-90a2-48e2-b48c-52e832483d9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:36:59.542059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:36:59.559564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5ece6-90a2-48e2-b48c-52e832483d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:36:59.561044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5ece6-90a2-48e2-b48c-52e832483d9a service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:36:59.577641 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1265/2547] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:01.832020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682697a-08aa-4881-ab78-897215542db4 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:01.843184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682697a-08aa-4881-ab78-897215542db4 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:01.847725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:01.849053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:01.849329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:01.849807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] PATH_INFO: `/v3/users/abe2a0e2a75e45a7a1ac301e5f7d3dca` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:01.850804 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe2a0e2a75e45a7a1ac301e5f7d3dca)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:01.852758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:01.903521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe2a0e2a75e45a7a1ac301e5f7d3dca'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:01.904204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe2a0e2a75e45a7a1ac301e5f7d3dca'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:01.906166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:01.915510 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe2a0e2a75e45a7a1ac301e5f7d3dca'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:01.923702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe2a0e2a75e45a7a1ac301e5f7d3dca'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:01.924339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe2a0e2a75e45a7a1ac301e5f7d3dca'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:01.944364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1283/2548] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:01 2022] DELETE /identity/v3/users/abe2a0e2a75e45a7a1ac301e5f7d3dca => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:01.949105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78773f37-41ba-4809-9507-d5abe420f074 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:01.949651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78773f37-41ba-4809-9507-d5abe420f074 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:01.950414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78773f37-41ba-4809-9507-d5abe420f074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:01.971624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-78773f37-41ba-4809-9507-d5abe420f074 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:01.973048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-78773f37-41ba-4809-9507-d5abe420f074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:02.043456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1266/2549] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:02.055047 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:02.056476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:02.140476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4cc1c-fe73-4cca-afaa-15e218142d73 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:02.141009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78773f37-41ba-4809-9507-d5abe420f074 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:02.142509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-78773f37-41ba-4809-9507-d5abe420f074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:02.144375 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d191c3f-c7b2-4163-8300-a6010093ac0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:02.145890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d191c3f-c7b2-4163-8300-a6010093ac0b service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:02.146835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d191c3f-c7b2-4163-8300-a6010093ac0b service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:02.147219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d191c3f-c7b2-4163-8300-a6010093ac0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:02.155257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78773f37-41ba-4809-9507-d5abe420f074 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:02.159975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d191c3f-c7b2-4163-8300-a6010093ac0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:02.162282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:02.163004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d191c3f-c7b2-4163-8300-a6010093ac0b service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:02.163134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:02.163594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:02.164086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] PATH_INFO: `/v3/users/9831a45115484d0680771badd8302ff4` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:02.165534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] RBAC: Authorizing `identity:delete_user(user_id=9831a45115484d0680771badd8302ff4)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:02.168000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:02.186695 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1284/2550] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:37:02.260651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9831a45115484d0680771badd8302ff4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.261468 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9831a45115484d0680771badd8302ff4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.287257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9831a45115484d0680771badd8302ff4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.298337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9831a45115484d0680771badd8302ff4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.311430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9831a45115484d0680771badd8302ff4'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.322579 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1267/2551] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:02 2022] DELETE /identity/v3/users/9831a45115484d0680771badd8302ff4 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:02.334367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dba9a-9619-45cb-a6f1-80c55cd7fa44 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:02.335062 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dba9a-9619-45cb-a6f1-80c55cd7fa44 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:02.335517 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dba9a-9619-45cb-a6f1-80c55cd7fa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:02.382985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d75dba9a-9619-45cb-a6f1-80c55cd7fa44 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:02.385644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d75dba9a-9619-45cb-a6f1-80c55cd7fa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:02.516816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1285/2552] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:02.534983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:02.540036 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:02.658297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dba9a-9619-45cb-a6f1-80c55cd7fa44 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:02.659940 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d75dba9a-9619-45cb-a6f1-80c55cd7fa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:02.673247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868c1fa-ad61-433e-83a2-a219272b63da admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:02.680894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152638c7-3aa0-424e-b95f-0f79d74b99a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:02.682637 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152638c7-3aa0-424e-b95f-0f79d74b99a1 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:02.683135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152638c7-3aa0-424e-b95f-0f79d74b99a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:02.684232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152638c7-3aa0-424e-b95f-0f79d74b99a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:02.685851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dba9a-9619-45cb-a6f1-80c55cd7fa44 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:02.696162 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152638c7-3aa0-424e-b95f-0f79d74b99a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:02.697863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152638c7-3aa0-424e-b95f-0f79d74b99a1 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:02.699521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:02.700112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:02.700409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:02.700678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] PATH_INFO: `/v3/projects/cfac65018f1545e08a4fdf92a158606d` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:02.703440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfac65018f1545e08a4fdf92a158606d)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:02.706745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:02.712566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1268/2553] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:02.788617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfac65018f1545e08a4fdf92a158606d', 'request_id': 'req-c9004d66-48f2-4b1a-a214-3bc426db65ba'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.804317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfac65018f1545e08a4fdf92a158606d', 'request_id': 'req-c9004d66-48f2-4b1a-a214-3bc426db65ba'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.812617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfac65018f1545e08a4fdf92a158606d', 'request_id': 'req-c9004d66-48f2-4b1a-a214-3bc426db65ba'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.814892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:INSERT=1,keystone:DELETE=9,keystone:SELECT=90 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:02.836543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] The token cache is being invalidate because project cfac65018f1545e08a4fdf92a158606d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:02.837104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.837660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c9004d66-48f2-4b1a-a214-3bc426db65ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:02.840285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1286/2554] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:02 2022] DELETE /identity/v3/projects/cfac65018f1545e08a4fdf92a158606d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:02.845374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff46ac1-d8e4-4ca6-8a8f-7eeaac3cf734 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:02.845374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff46ac1-d8e4-4ca6-8a8f-7eeaac3cf734 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:02.845614 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff46ac1-d8e4-4ca6-8a8f-7eeaac3cf734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:02.855543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:02.883561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-3ff46ac1-d8e4-4ca6-8a8f-7eeaac3cf734 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:02.886402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3ff46ac1-d8e4-4ca6-8a8f-7eeaac3cf734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:02.965255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1269/2555] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:02.974087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbc5a3-4540-4c2b-8ef3-68c94ff97bba None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:02.974611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbc5a3-4540-4c2b-8ef3-68c94ff97bba None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:02.974973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbc5a3-4540-4c2b-8ef3-68c94ff97bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:02.996840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3ddbc5a3-4540-4c2b-8ef3-68c94ff97bba None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:02.998007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3ddbc5a3-4540-4c2b-8ef3-68c94ff97bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:03.006730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1287/2556] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:03.010913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff46ac1-d8e4-4ca6-8a8f-7eeaac3cf734 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:03.021379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff46ac1-d8e4-4ca6-8a8f-7eeaac3cf734 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:03.026024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21bbd3-d906-4c8e-8227-a747153a84b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:03.026655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21bbd3-d906-4c8e-8227-a747153a84b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.026932 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21bbd3-d906-4c8e-8227-a747153a84b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.027199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21bbd3-d906-4c8e-8227-a747153a84b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.027846 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21bbd3-d906-4c8e-8227-a747153a84b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:37:03.028145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21bbd3-d906-4c8e-8227-a747153a84b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:03.029793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21bbd3-d906-4c8e-8227-a747153a84b4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:03.042596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1270/2557] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:37:03.046878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbc5a3-4540-4c2b-8ef3-68c94ff97bba None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:03.057350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbc5a3-4540-4c2b-8ef3-68c94ff97bba None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:03.061950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337bd70-632c-4aaa-af54-883f1abe6f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:03.062560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337bd70-632c-4aaa-af54-883f1abe6f9d admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.062833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337bd70-632c-4aaa-af54-883f1abe6f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.063102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337bd70-632c-4aaa-af54-883f1abe6f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.063745 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337bd70-632c-4aaa-af54-883f1abe6f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:03.065735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337bd70-632c-4aaa-af54-883f1abe6f9d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:03.097570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1288/2558] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:03 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:03.099478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21bbd3-d906-4c8e-8227-a747153a84b4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:03.109595 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21bbd3-d906-4c8e-8227-a747153a84b4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:03.168013 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cd4d8-48b5-45cf-9089-0091176c01b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:03.168662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cd4d8-48b5-45cf-9089-0091176c01b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.168937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cd4d8-48b5-45cf-9089-0091176c01b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.169206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462cd4d8-48b5-45cf-9089-0091176c01b6 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.169834 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cd4d8-48b5-45cf-9089-0091176c01b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:03.171934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462cd4d8-48b5-45cf-9089-0091176c01b6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:03.194092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1271/2559] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:03 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:03.198435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337bd70-632c-4aaa-af54-883f1abe6f9d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:03.210045 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337bd70-632c-4aaa-af54-883f1abe6f9d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:03.214888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec980d-beae-40fd-9eb2-005ceb39516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:03.215609 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec980d-beae-40fd-9eb2-005ceb39516c admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.216044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec980d-beae-40fd-9eb2-005ceb39516c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.216423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec980d-beae-40fd-9eb2-005ceb39516c admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.217205 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec980d-beae-40fd-9eb2-005ceb39516c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:03.218900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec980d-beae-40fd-9eb2-005ceb39516c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:03.230428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1289/2560] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:03.234564 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cd4d8-48b5-45cf-9089-0091176c01b6 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:03.246198 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462cd4d8-48b5-45cf-9089-0091176c01b6 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:03.251418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ecacf-dcf5-4358-ab8d-a922fcb010dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:03.252445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ecacf-dcf5-4358-ab8d-a922fcb010dc admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.252445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ecacf-dcf5-4358-ab8d-a922fcb010dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.252717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ecacf-dcf5-4358-ab8d-a922fcb010dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.253233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ecacf-dcf5-4358-ab8d-a922fcb010dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:03.255089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ecacf-dcf5-4358-ab8d-a922fcb010dc admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:03.266914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1272/2561] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:03.271856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec980d-beae-40fd-9eb2-005ceb39516c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:03.282253 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec980d-beae-40fd-9eb2-005ceb39516c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:03.288012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4d6e2-acdf-4451-9f0e-aa15935b449d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:03.288837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4d6e2-acdf-4451-9f0e-aa15935b449d admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.289145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4d6e2-acdf-4451-9f0e-aa15935b449d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.289453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4d6e2-acdf-4451-9f0e-aa15935b449d admin admin] PATH_INFO: `/v3/projects/07af35e80c004621bd72ab7336930693/users/7d042aa1a703464eab4a440db9ce1374/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.306586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4d6e2-acdf-4451-9f0e-aa15935b449d admin admin] RBAC: Authorizing `identity:create_grant(project_id=07af35e80c004621bd72ab7336930693, user_id=7d042aa1a703464eab4a440db9ce1374, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:03.309316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4d6e2-acdf-4451-9f0e-aa15935b449d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:03.323129 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1290/2562] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:03 2022] PUT /identity/v3/projects/07af35e80c004621bd72ab7336930693/users/7d042aa1a703464eab4a440db9ce1374/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:03.328639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11cfd-f60a-4344-a0b1-103556c32ac6 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.329099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11cfd-f60a-4344-a0b1-103556c32ac6 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.329395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11cfd-f60a-4344-a0b1-103556c32ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.351704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-eaa11cfd-f60a-4344-a0b1-103556c32ac6 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:03.353171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-eaa11cfd-f60a-4344-a0b1-103556c32ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:03.426462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1273/2563] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:03.435855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4d6e2-acdf-4451-9f0e-aa15935b449d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:03.437747 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-59d4d6e2-acdf-4451-9f0e-aa15935b449d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:03.505387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4d6e2-acdf-4451-9f0e-aa15935b449d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:03.509133 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552107cc-ddcf-4cb2-8f40-e486910e95f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:03.509751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552107cc-ddcf-4cb2-8f40-e486910e95f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.510029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552107cc-ddcf-4cb2-8f40-e486910e95f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.510301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552107cc-ddcf-4cb2-8f40-e486910e95f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.520775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552107cc-ddcf-4cb2-8f40-e486910e95f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:03.522229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552107cc-ddcf-4cb2-8f40-e486910e95f6 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:03.532305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1291/2564] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:03.557922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11cfd-f60a-4344-a0b1-103556c32ac6 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:03.561556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-eaa11cfd-f60a-4344-a0b1-103556c32ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:03.576157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11cfd-f60a-4344-a0b1-103556c32ac6 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:03.581287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:03.582105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.582254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.583598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] PATH_INFO: `/v3/users/ecf29db866eb43fa954b0d440577f45f` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.585030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf29db866eb43fa954b0d440577f45f)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:03.587136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:03.601982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:03.660090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf29db866eb43fa954b0d440577f45f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:03.661562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf29db866eb43fa954b0d440577f45f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:03.678105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf29db866eb43fa954b0d440577f45f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:03.689556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf29db866eb43fa954b0d440577f45f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:03.700898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf29db866eb43fa954b0d440577f45f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:03.713477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1274/2565] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:03 2022] DELETE /identity/v3/users/ecf29db866eb43fa954b0d440577f45f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:03.814324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552107cc-ddcf-4cb2-8f40-e486910e95f6 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:03.815886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-552107cc-ddcf-4cb2-8f40-e486910e95f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:03.829366 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552107cc-ddcf-4cb2-8f40-e486910e95f6 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:03.908462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:03.910137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:03.910390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:03.910710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] PATH_INFO: `/v3/projects/c035d676c6f64b7fb4d7a4039e3d86b1` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:03.912138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c035d676c6f64b7fb4d7a4039e3d86b1)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:03.914554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:03.960800 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:03.980617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c035d676c6f64b7fb4d7a4039e3d86b1', 'request_id': 'req-1513e095-db57-4a6b-8f1b-0d29b91240c7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:03.989240 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c035d676c6f64b7fb4d7a4039e3d86b1', 'request_id': 'req-1513e095-db57-4a6b-8f1b-0d29b91240c7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:03.997525 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c035d676c6f64b7fb4d7a4039e3d86b1', 'request_id': 'req-1513e095-db57-4a6b-8f1b-0d29b91240c7'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:04.012268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] The token cache is being invalidate because project c035d676c6f64b7fb4d7a4039e3d86b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:04.012996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:04.013749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1513e095-db57-4a6b-8f1b-0d29b91240c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:04.018669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1292/2566] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:03 2022] DELETE /identity/v3/projects/c035d676c6f64b7fb4d7a4039e3d86b1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:04.116118 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:04.119275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:04.149532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4301c78-07cf-4b84-a277-daa6a03a31c7 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:04.316807 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:04.318333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:04.318780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:04.319061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] PATH_INFO: `/v3/projects/06942912e37346eaa2310a56b84f3886` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:04.320431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06942912e37346eaa2310a56b84f3886)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:04.322553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:04.382107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06942912e37346eaa2310a56b84f3886', 'request_id': 'req-7a4c8cda-971e-4da3-9159-4a81eebeffd9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:04.389662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06942912e37346eaa2310a56b84f3886', 'request_id': 'req-7a4c8cda-971e-4da3-9159-4a81eebeffd9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:04.400659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06942912e37346eaa2310a56b84f3886', 'request_id': 'req-7a4c8cda-971e-4da3-9159-4a81eebeffd9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:04.423635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] The token cache is being invalidate because project 06942912e37346eaa2310a56b84f3886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:04.423937 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:04.424887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-7a4c8cda-971e-4da3-9159-4a81eebeffd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:04.426626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1275/2567] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:04 2022] DELETE /identity/v3/projects/06942912e37346eaa2310a56b84f3886 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:04.431867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51514279-8e4d-4921-b94c-6c19f939b78c None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:04.432541 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51514279-8e4d-4921-b94c-6c19f939b78c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:04.433070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51514279-8e4d-4921-b94c-6c19f939b78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:04.455635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-51514279-8e4d-4921-b94c-6c19f939b78c None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:04.457021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-51514279-8e4d-4921-b94c-6c19f939b78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:04.548694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1293/2568] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:04.553775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b82f6f-8763-43ed-9959-608c451e9db0 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:04.556729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b82f6f-8763-43ed-9959-608c451e9db0 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:04.557062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b82f6f-8763-43ed-9959-608c451e9db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:04.581975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-d9b82f6f-8763-43ed-9959-608c451e9db0 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:04.584218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d9b82f6f-8763-43ed-9959-608c451e9db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:04.600438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51514279-8e4d-4921-b94c-6c19f939b78c None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:04.600599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1276/2569] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:04.611714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd7b82-1b82-472c-9058-efb5175585d5 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:04.612055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd7b82-1b82-472c-9058-efb5175585d5 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:04.612250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd7b82-1b82-472c-9058-efb5175585d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:04.614455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51514279-8e4d-4921-b94c-6c19f939b78c None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:04.624166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad26d3d-b713-4bb5-86a7-0e86b1bae5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:04.625144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad26d3d-b713-4bb5-86a7-0e86b1bae5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:04.625659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:04.625789 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad26d3d-b713-4bb5-86a7-0e86b1bae5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:04.626256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad26d3d-b713-4bb5-86a7-0e86b1bae5f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:04.627127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad26d3d-b713-4bb5-86a7-0e86b1bae5f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:37:04.627653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad26d3d-b713-4bb5-86a7-0e86b1bae5f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:04.630632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad26d3d-b713-4bb5-86a7-0e86b1bae5f8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:04.660489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1294/2570] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:37:04.665578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad26d3d-b713-4bb5-86a7-0e86b1bae5f8 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:04.690103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-d2cd7b82-1b82-472c-9058-efb5175585d5 None None] MFA Rules not processed for user `7d042aa1a703464eab4a440db9ce1374`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:04.695151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d2cd7b82-1b82-472c-9058-efb5175585d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:04.711794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad26d3d-b713-4bb5-86a7-0e86b1bae5f8 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:04.724717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa45012-4c83-457e-bd34-c3eba8ebe3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:04.725353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa45012-4c83-457e-bd34-c3eba8ebe3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:04.726192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa45012-4c83-457e-bd34-c3eba8ebe3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:04.726192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa45012-4c83-457e-bd34-c3eba8ebe3a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:04.726495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa45012-4c83-457e-bd34-c3eba8ebe3a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:04.728627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa45012-4c83-457e-bd34-c3eba8ebe3a4 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:04.792895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1295/2571] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:04 2022] POST /identity/v3/projects => generated 403 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:37:04.796610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa45012-4c83-457e-bd34-c3eba8ebe3a4 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:04.838053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1277/2572] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:37:04.847290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd7b82-1b82-472c-9058-efb5175585d5 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:04.848851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d2cd7b82-1b82-472c-9058-efb5175585d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:04.887016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa45012-4c83-457e-bd34-c3eba8ebe3a4 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:04.892127 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164ad21-eb24-46bd-9186-c61501e82d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:04.892799 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164ad21-eb24-46bd-9186-c61501e82d0b admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:04.893080 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164ad21-eb24-46bd-9186-c61501e82d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:04.893353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164ad21-eb24-46bd-9186-c61501e82d0b admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:04.893991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164ad21-eb24-46bd-9186-c61501e82d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:04.896158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164ad21-eb24-46bd-9186-c61501e82d0b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:04.938195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1296/2573] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:04 2022] POST /identity/v3/users => generated 302 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:04.942811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164ad21-eb24-46bd-9186-c61501e82d0b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:04.945740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd7b82-1b82-472c-9058-efb5175585d5 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:04.956284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164ad21-eb24-46bd-9186-c61501e82d0b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:04.962258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e783a-8651-4bef-be01-364574eca23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:04.963006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e783a-8651-4bef-be01-364574eca23b admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:04.963358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e783a-8651-4bef-be01-364574eca23b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:04.963700 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e783a-8651-4bef-be01-364574eca23b admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:04.964461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e783a-8651-4bef-be01-364574eca23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:04.966212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e783a-8651-4bef-be01-364574eca23b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:04.981124 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1297/2574] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:37:04.982390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e783a-8651-4bef-be01-364574eca23b admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:04.989429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:04.995247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e783a-8651-4bef-be01-364574eca23b admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:05.001572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bab2d1-0303-4ecb-9927-6f2c026cad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.002631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab2d1-0303-4ecb-9927-6f2c026cad41 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.003057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab2d1-0303-4ecb-9927-6f2c026cad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.003618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab2d1-0303-4ecb-9927-6f2c026cad41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.004890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bab2d1-0303-4ecb-9927-6f2c026cad41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.006684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bab2d1-0303-4ecb-9927-6f2c026cad41 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.015309 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfca22-3cd6-40ff-96ab-0ade680247d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.015930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfca22-3cd6-40ff-96ab-0ade680247d3 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.016211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfca22-3cd6-40ff-96ab-0ade680247d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.016961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dfca22-3cd6-40ff-96ab-0ade680247d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.019042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1298/2575] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:05.023439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bab2d1-0303-4ecb-9927-6f2c026cad41 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:05.037487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bab2d1-0303-4ecb-9927-6f2c026cad41 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:05.042289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e4872-6db3-4293-96a8-15c6e69b92eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.043179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e4872-6db3-4293-96a8-15c6e69b92eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.043435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e4872-6db3-4293-96a8-15c6e69b92eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.043716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e4872-6db3-4293-96a8-15c6e69b92eb admin admin] PATH_INFO: `/v3/projects/e81cb920413545f4b0016712292d27f5/users/13d06a05e63541f1a71420fc09551933/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.063320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e4872-6db3-4293-96a8-15c6e69b92eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81cb920413545f4b0016712292d27f5, user_id=13d06a05e63541f1a71420fc09551933, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.070153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e4872-6db3-4293-96a8-15c6e69b92eb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.079063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfca22-3cd6-40ff-96ab-0ade680247d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.080411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dfca22-3cd6-40ff-96ab-0ade680247d3 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.087282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1299/2576] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:05 2022] PUT /identity/v3/projects/e81cb920413545f4b0016712292d27f5/users/13d06a05e63541f1a71420fc09551933/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:37:05.093018 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1278/2577] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:05.097468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049af45-eb41-440f-959c-6407ba669a81 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.097763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049af45-eb41-440f-959c-6407ba669a81 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.098577 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049af45-eb41-440f-959c-6407ba669a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.134579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d049af45-eb41-440f-959c-6407ba669a81 None None] MFA Rules not processed for user `13d06a05e63541f1a71420fc09551933`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:05.135816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d049af45-eb41-440f-959c-6407ba669a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:05.229419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1300/2578] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:05 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:05.232858 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfca22-3cd6-40ff-96ab-0ade680247d3 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:05.240957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:05.246598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dfca22-3cd6-40ff-96ab-0ade680247d3 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:05.348429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331a61c-6145-47bb-a562-897a91f8413f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.349133 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331a61c-6145-47bb-a562-897a91f8413f admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.349409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331a61c-6145-47bb-a562-897a91f8413f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.349684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331a61c-6145-47bb-a562-897a91f8413f admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.350375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331a61c-6145-47bb-a562-897a91f8413f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.352627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331a61c-6145-47bb-a562-897a91f8413f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.384095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1279/2579] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:05 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:37:05.388188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049af45-eb41-440f-959c-6407ba669a81 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:05.414160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049af45-eb41-440f-959c-6407ba669a81 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:05.497113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc09d9-21b1-4e68-9ca8-fb3deb94ad00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.497892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc09d9-21b1-4e68-9ca8-fb3deb94ad00 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.498286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc09d9-21b1-4e68-9ca8-fb3deb94ad00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.498661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc09d9-21b1-4e68-9ca8-fb3deb94ad00 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.499398 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc09d9-21b1-4e68-9ca8-fb3deb94ad00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.501594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc09d9-21b1-4e68-9ca8-fb3deb94ad00 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.524281 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1301/2580] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:05 2022] POST /identity/v3/users => generated 309 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:05.528653 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331a61c-6145-47bb-a562-897a91f8413f admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:05.539436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331a61c-6145-47bb-a562-897a91f8413f admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:05.544360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa89d9-f385-412f-92f5-5e01b14c0627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.545157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa89d9-f385-412f-92f5-5e01b14c0627 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.545496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa89d9-f385-412f-92f5-5e01b14c0627 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.545684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa89d9-f385-412f-92f5-5e01b14c0627 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.546284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa89d9-f385-412f-92f5-5e01b14c0627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.547905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa89d9-f385-412f-92f5-5e01b14c0627 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.558866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1280/2581] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:37:05.563075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc09d9-21b1-4e68-9ca8-fb3deb94ad00 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:05.573668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc09d9-21b1-4e68-9ca8-fb3deb94ad00 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:05.581594 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c5d3c-a5bf-4f6e-8e3f-39e224533e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.582351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5d3c-a5bf-4f6e-8e3f-39e224533e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.583276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5d3c-a5bf-4f6e-8e3f-39e224533e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.583809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5d3c-a5bf-4f6e-8e3f-39e224533e49 admin admin] PATH_INFO: `/v3/projects/fdd8a16d957e4eb3b40a4b890729347d/users/676f1037704441af836ba66f90b44530/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.601038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c5d3c-a5bf-4f6e-8e3f-39e224533e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd8a16d957e4eb3b40a4b890729347d, user_id=676f1037704441af836ba66f90b44530, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.603728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c5d3c-a5bf-4f6e-8e3f-39e224533e49 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.617303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1302/2582] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:05 2022] PUT /identity/v3/projects/fdd8a16d957e4eb3b40a4b890729347d/users/676f1037704441af836ba66f90b44530/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:05.621431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa89d9-f385-412f-92f5-5e01b14c0627 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:05.632384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa89d9-f385-412f-92f5-5e01b14c0627 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:05.689914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fa35a-67cb-4ad8-bbd3-74811fcfb17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.690647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fa35a-67cb-4ad8-bbd3-74811fcfb17b admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.690980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fa35a-67cb-4ad8-bbd3-74811fcfb17b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.691317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fa35a-67cb-4ad8-bbd3-74811fcfb17b admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.692197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fa35a-67cb-4ad8-bbd3-74811fcfb17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.693868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fa35a-67cb-4ad8-bbd3-74811fcfb17b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.703982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1281/2583] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:05.708792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c5d3c-a5bf-4f6e-8e3f-39e224533e49 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:05.719178 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c5d3c-a5bf-4f6e-8e3f-39e224533e49 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:05.725069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe4392-c664-45de-ac71-1e7822d98bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.725737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe4392-c664-45de-ac71-1e7822d98bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.726018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe4392-c664-45de-ac71-1e7822d98bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.726294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe4392-c664-45de-ac71-1e7822d98bf3 admin admin] PATH_INFO: `/v3/projects/fdd8a16d957e4eb3b40a4b890729347d/users/676f1037704441af836ba66f90b44530/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.728364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe4392-c664-45de-ac71-1e7822d98bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd8a16d957e4eb3b40a4b890729347d, user_id=676f1037704441af836ba66f90b44530, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.731179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe4392-c664-45de-ac71-1e7822d98bf3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.742404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1303/2584] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:05 2022] PUT /identity/v3/projects/fdd8a16d957e4eb3b40a4b890729347d/users/676f1037704441af836ba66f90b44530/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:05.754197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7262d-f445-42a8-8af2-2d49648b0410 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.755070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7262d-f445-42a8-8af2-2d49648b0410 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.755325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7262d-f445-42a8-8af2-2d49648b0410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.789202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-29a7262d-f445-42a8-8af2-2d49648b0410 None None] MFA Rules not processed for user `676f1037704441af836ba66f90b44530`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:05.790833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-29a7262d-f445-42a8-8af2-2d49648b0410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:05.862136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:05.873201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1282/2585] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:05 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:05.878213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe4392-c664-45de-ac71-1e7822d98bf3 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:05.894422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:05.940708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe4392-c664-45de-ac71-1e7822d98bf3 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:05.945374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba41a38-3635-4503-a75e-8ef359ec7cc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:05.945994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba41a38-3635-4503-a75e-8ef359ec7cc9 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:05.946269 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba41a38-3635-4503-a75e-8ef359ec7cc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:05.946672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba41a38-3635-4503-a75e-8ef359ec7cc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:05.958149 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba41a38-3635-4503-a75e-8ef359ec7cc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:05.959831 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba41a38-3635-4503-a75e-8ef359ec7cc9 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:05.976207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1304/2586] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:05 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:06.143230 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7262d-f445-42a8-8af2-2d49648b0410 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:06.229291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7262d-f445-42a8-8af2-2d49648b0410 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:06.242655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:06.243522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:06.243867 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:06.244201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] PATH_INFO: `/v3/users/13d06a05e63541f1a71420fc09551933` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:06.245478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d06a05e63541f1a71420fc09551933)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:06.247508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:06.296567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d06a05e63541f1a71420fc09551933'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:06.297234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d06a05e63541f1a71420fc09551933'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:06.309531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d06a05e63541f1a71420fc09551933'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:06.316517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d06a05e63541f1a71420fc09551933'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:06.324544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d06a05e63541f1a71420fc09551933'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:06.330706 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1283/2587] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:06 2022] DELETE /identity/v3/users/13d06a05e63541f1a71420fc09551933 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:06.335751 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb06065-01b1-4f0e-8801-b2a46c616f5d None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:06.336041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb06065-01b1-4f0e-8801-b2a46c616f5d None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:06.336310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb06065-01b1-4f0e-8801-b2a46c616f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:06.355722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-edb06065-01b1-4f0e-8801-b2a46c616f5d None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:06.356959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-edb06065-01b1-4f0e-8801-b2a46c616f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:06.416583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1305/2588] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:06.426470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:06.427890 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:06.486639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321681b6-5b8d-41ca-9070-4783c2256e98 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:06.492317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5b877-f81d-43e9-9acd-16b51634a5f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:06.492979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5b877-f81d-43e9-9acd-16b51634a5f7 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:06.493649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5b877-f81d-43e9-9acd-16b51634a5f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:06.493649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5b877-f81d-43e9-9acd-16b51634a5f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:06.503289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5b877-f81d-43e9-9acd-16b51634a5f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:06.505334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5b877-f81d-43e9-9acd-16b51634a5f7 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:06.517438 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1284/2589] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:06.562117 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f84fa-30c6-4b4f-9bdc-83a9b22a46c6 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:06.562538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f84fa-30c6-4b4f-9bdc-83a9b22a46c6 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:06.562976 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f84fa-30c6-4b4f-9bdc-83a9b22a46c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:06.586294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-733f84fa-30c6-4b4f-9bdc-83a9b22a46c6 None None] MFA Rules not processed for user `7d042aa1a703464eab4a440db9ce1374`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:06.587759 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-733f84fa-30c6-4b4f-9bdc-83a9b22a46c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:06.657514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1306/2590] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:07.192546 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5b877-f81d-43e9-9acd-16b51634a5f7 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:07.194157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-dca5b877-f81d-43e9-9acd-16b51634a5f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:07.207337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5b877-f81d-43e9-9acd-16b51634a5f7 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:07.212153 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:07.212813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:07.213092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:07.213365 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] PATH_INFO: `/v3/projects/e81cb920413545f4b0016712292d27f5` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:07.214501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81cb920413545f4b0016712292d27f5)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:07.216434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:07.238246 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:07.273360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81cb920413545f4b0016712292d27f5', 'request_id': 'req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.279176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81cb920413545f4b0016712292d27f5', 'request_id': 'req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.286360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81cb920413545f4b0016712292d27f5', 'request_id': 'req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.299243 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] The token cache is being invalidate because project e81cb920413545f4b0016712292d27f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:07.299533 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.300182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.301774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1285/2591] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:07 2022] DELETE /identity/v3/projects/e81cb920413545f4b0016712292d27f5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:07.305699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f84fa-30c6-4b4f-9bdc-83a9b22a46c6 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:07.316273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f84fa-30c6-4b4f-9bdc-83a9b22a46c6 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:07.373401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:07.374031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:07.374301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:07.374568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] PATH_INFO: `/v3/users/676f1037704441af836ba66f90b44530` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:07.375675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=676f1037704441af836ba66f90b44530)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:07.377641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:07.379855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:07.428648 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '676f1037704441af836ba66f90b44530'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.429394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '676f1037704441af836ba66f90b44530'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.436971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '676f1037704441af836ba66f90b44530'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.444362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '676f1037704441af836ba66f90b44530'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.445036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-4fcf98fa-1ca0-4c8a-a9a4-003b1bb38b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '676f1037704441af836ba66f90b44530'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.462791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1307/2592] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:07 2022] DELETE /identity/v3/users/676f1037704441af836ba66f90b44530 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:37:07.772803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:07.774798 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:07.794587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538164b3-09ec-4e72-bed1-d24e4c4f2c6f admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:07.867561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:07.868460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:07.868991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:07.869354 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] PATH_INFO: `/v3/projects/fdd8a16d957e4eb3b40a4b890729347d` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:07.871069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd8a16d957e4eb3b40a4b890729347d)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:07.873144 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:07.945300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd8a16d957e4eb3b40a4b890729347d', 'request_id': 'req-e1d05602-3998-4859-8bfb-2e721c705f88'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.952105 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd8a16d957e4eb3b40a4b890729347d', 'request_id': 'req-e1d05602-3998-4859-8bfb-2e721c705f88'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.959423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd8a16d957e4eb3b40a4b890729347d', 'request_id': 'req-e1d05602-3998-4859-8bfb-2e721c705f88'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.972856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] The token cache is being invalidate because project fdd8a16d957e4eb3b40a4b890729347d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:07.973694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.974506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e1d05602-3998-4859-8bfb-2e721c705f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:07.976110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1286/2593] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:07 2022] DELETE /identity/v3/projects/fdd8a16d957e4eb3b40a4b890729347d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:07.982782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e9fe2-75f2-4942-95bc-110979a1f010 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:07.983153 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e9fe2-75f2-4942-95bc-110979a1f010 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:07.983494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e9fe2-75f2-4942-95bc-110979a1f010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.011861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-109e9fe2-75f2-4942-95bc-110979a1f010 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:08.013458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-109e9fe2-75f2-4942-95bc-110979a1f010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:08.101890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1308/2594] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:08.109589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c5b04-9e3f-4389-8142-83fc57c7dbfe None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.109880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c5b04-9e3f-4389-8142-83fc57c7dbfe None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.110159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c5b04-9e3f-4389-8142-83fc57c7dbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.130704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-861c5b04-9e3f-4389-8142-83fc57c7dbfe None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:08.132093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-861c5b04-9e3f-4389-8142-83fc57c7dbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:08.143402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1287/2595] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:08.144598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e9fe2-75f2-4942-95bc-110979a1f010 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.155391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e9fe2-75f2-4942-95bc-110979a1f010 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.161823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e8e32-a73e-476b-8af5-4c6cd31d71e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.162704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e8e32-a73e-476b-8af5-4c6cd31d71e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.163073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e8e32-a73e-476b-8af5-4c6cd31d71e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.163422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e8e32-a73e-476b-8af5-4c6cd31d71e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.164157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e8e32-a73e-476b-8af5-4c6cd31d71e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:37:08.164545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e8e32-a73e-476b-8af5-4c6cd31d71e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.166168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e8e32-a73e-476b-8af5-4c6cd31d71e6 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.179493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1309/2596] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:37:08.184033 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c5b04-9e3f-4389-8142-83fc57c7dbfe None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.194457 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c5b04-9e3f-4389-8142-83fc57c7dbfe None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.199687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815b555-bc68-48dc-bfbb-3fc11d52db8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.200297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815b555-bc68-48dc-bfbb-3fc11d52db8c admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.200605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815b555-bc68-48dc-bfbb-3fc11d52db8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.200875 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815b555-bc68-48dc-bfbb-3fc11d52db8c admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.201513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815b555-bc68-48dc-bfbb-3fc11d52db8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.203435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815b555-bc68-48dc-bfbb-3fc11d52db8c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.235163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1288/2597] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:08 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:08.240272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e8e32-a73e-476b-8af5-4c6cd31d71e6 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.257674 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e8e32-a73e-476b-8af5-4c6cd31d71e6 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.315607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61d8b6-e868-4df3-b5a4-a8c9064be1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.316221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61d8b6-e868-4df3-b5a4-a8c9064be1df admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.316470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61d8b6-e868-4df3-b5a4-a8c9064be1df admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.316869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61d8b6-e868-4df3-b5a4-a8c9064be1df admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.317496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61d8b6-e868-4df3-b5a4-a8c9064be1df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.319417 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61d8b6-e868-4df3-b5a4-a8c9064be1df admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.330572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:08.342549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1310/2598] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:08 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:37:08.346620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815b555-bc68-48dc-bfbb-3fc11d52db8c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.357993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815b555-bc68-48dc-bfbb-3fc11d52db8c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.362506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75eccec-4181-43fe-ab99-4e046229d975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.363030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75eccec-4181-43fe-ab99-4e046229d975 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.363304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75eccec-4181-43fe-ab99-4e046229d975 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.363634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75eccec-4181-43fe-ab99-4e046229d975 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.364279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75eccec-4181-43fe-ab99-4e046229d975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.366126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75eccec-4181-43fe-ab99-4e046229d975 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.367352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:08.377264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1289/2599] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:08.380950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61d8b6-e868-4df3-b5a4-a8c9064be1df admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.390881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61d8b6-e868-4df3-b5a4-a8c9064be1df admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.396137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f605a-3ec5-49f7-9c8e-298e45cc02a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.396767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f605a-3ec5-49f7-9c8e-298e45cc02a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.397056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f605a-3ec5-49f7-9c8e-298e45cc02a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.397332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f605a-3ec5-49f7-9c8e-298e45cc02a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.397975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f605a-3ec5-49f7-9c8e-298e45cc02a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.399576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f605a-3ec5-49f7-9c8e-298e45cc02a3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.409362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1311/2600] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:37:08.414552 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75eccec-4181-43fe-ab99-4e046229d975 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.424354 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75eccec-4181-43fe-ab99-4e046229d975 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.429397 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6dd148-3bee-4531-a194-74ea85cf078b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.430040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd148-3bee-4531-a194-74ea85cf078b admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.430322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd148-3bee-4531-a194-74ea85cf078b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.430596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd148-3bee-4531-a194-74ea85cf078b admin admin] PATH_INFO: `/v3/projects/021943111d844fe093495b2bfbc4ecdc/users/d01a9a05c17a49a5976ec11cc3b467fd/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.445356 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dd148-3bee-4531-a194-74ea85cf078b admin admin] RBAC: Authorizing `identity:create_grant(project_id=021943111d844fe093495b2bfbc4ecdc, user_id=d01a9a05c17a49a5976ec11cc3b467fd, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.447989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dd148-3bee-4531-a194-74ea85cf078b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.461153 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1290/2601] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:08 2022] PUT /identity/v3/projects/021943111d844fe093495b2bfbc4ecdc/users/d01a9a05c17a49a5976ec11cc3b467fd/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:08.471367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fddae6-ca85-4035-93f0-1e4b1a28a97e None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.471656 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fddae6-ca85-4035-93f0-1e4b1a28a97e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.471961 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fddae6-ca85-4035-93f0-1e4b1a28a97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.501642 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-86fddae6-ca85-4035-93f0-1e4b1a28a97e None None] MFA Rules not processed for user `d01a9a05c17a49a5976ec11cc3b467fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:08.502885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-86fddae6-ca85-4035-93f0-1e4b1a28a97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:08.566473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1312/2602] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:37:08.571088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6dd148-3bee-4531-a194-74ea85cf078b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.637570 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6dd148-3bee-4531-a194-74ea85cf078b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.642989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381a9e0-b6d7-494d-9c89-6795646b9ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.643850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381a9e0-b6d7-494d-9c89-6795646b9ffb admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.644123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381a9e0-b6d7-494d-9c89-6795646b9ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.644427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381a9e0-b6d7-494d-9c89-6795646b9ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.645670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381a9e0-b6d7-494d-9c89-6795646b9ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.647808 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381a9e0-b6d7-494d-9c89-6795646b9ffb admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.684842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1291/2603] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:08 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:08.688571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fddae6-ca85-4035-93f0-1e4b1a28a97e None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.704821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fddae6-ca85-4035-93f0-1e4b1a28a97e None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.799296 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4789073-c5ed-4ef2-8ea8-753a405db696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.799908 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4789073-c5ed-4ef2-8ea8-753a405db696 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.800182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4789073-c5ed-4ef2-8ea8-753a405db696 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.800449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4789073-c5ed-4ef2-8ea8-753a405db696 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.801916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4789073-c5ed-4ef2-8ea8-753a405db696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.803927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4789073-c5ed-4ef2-8ea8-753a405db696 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.828053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1313/2604] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:08 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:37:08.832150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381a9e0-b6d7-494d-9c89-6795646b9ffb admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.843834 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381a9e0-b6d7-494d-9c89-6795646b9ffb admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.849860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea400a-2a10-4229-941d-43565c0d29cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.850297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea400a-2a10-4229-941d-43565c0d29cc admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.850655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea400a-2a10-4229-941d-43565c0d29cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.851086 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea400a-2a10-4229-941d-43565c0d29cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.851997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea400a-2a10-4229-941d-43565c0d29cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.853925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea400a-2a10-4229-941d-43565c0d29cc admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.866112 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1292/2605] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:08.870160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4789073-c5ed-4ef2-8ea8-753a405db696 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.883198 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4789073-c5ed-4ef2-8ea8-753a405db696 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.888997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517da7d9-9f59-4a24-ad4a-a3761d693411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.889545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517da7d9-9f59-4a24-ad4a-a3761d693411 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.889823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517da7d9-9f59-4a24-ad4a-a3761d693411 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.890093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517da7d9-9f59-4a24-ad4a-a3761d693411 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.890735 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517da7d9-9f59-4a24-ad4a-a3761d693411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.892383 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517da7d9-9f59-4a24-ad4a-a3761d693411 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.906181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1314/2606] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:08.907342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea400a-2a10-4229-941d-43565c0d29cc admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:08.917883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea400a-2a10-4229-941d-43565c0d29cc admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:08.922477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fb9cc-7a76-483e-86c8-85e1f1ff4b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:08.923128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fb9cc-7a76-483e-86c8-85e1f1ff4b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.923406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fb9cc-7a76-483e-86c8-85e1f1ff4b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.923677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fb9cc-7a76-483e-86c8-85e1f1ff4b3e admin admin] PATH_INFO: `/v3/projects/79469619f69148d99e2924112931581b/users/75d7afe46648443bbca3869e75657190/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.942238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fb9cc-7a76-483e-86c8-85e1f1ff4b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=79469619f69148d99e2924112931581b, user_id=75d7afe46648443bbca3869e75657190, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:08.945106 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fb9cc-7a76-483e-86c8-85e1f1ff4b3e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:08.958439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1293/2607] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:08 2022] PUT /identity/v3/projects/79469619f69148d99e2924112931581b/users/75d7afe46648443bbca3869e75657190/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:08.970145 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2163b2-4590-4fc6-a9f4-3fa2a9d73b24 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:08.970438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2163b2-4590-4fc6-a9f4-3fa2a9d73b24 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:08.970765 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2163b2-4590-4fc6-a9f4-3fa2a9d73b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:08.999712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-9b2163b2-4590-4fc6-a9f4-3fa2a9d73b24 None None] MFA Rules not processed for user `75d7afe46648443bbca3869e75657190`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:09.000950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-9b2163b2-4590-4fc6-a9f4-3fa2a9d73b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:09.043872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:09.072389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1315/2608] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:09.081781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fb9cc-7a76-483e-86c8-85e1f1ff4b3e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:09.163472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fb9cc-7a76-483e-86c8-85e1f1ff4b3e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:09.168543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdfb63-5867-41dd-a26e-120146f78ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:09.169283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdfb63-5867-41dd-a26e-120146f78ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:09.169624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdfb63-5867-41dd-a26e-120146f78ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:09.169957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdfb63-5867-41dd-a26e-120146f78ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:09.170675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdfb63-5867-41dd-a26e-120146f78ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:09.172743 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdfb63-5867-41dd-a26e-120146f78ca5 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:09.206124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1294/2609] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:09 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:09.211526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2163b2-4590-4fc6-a9f4-3fa2a9d73b24 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:09.224909 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2163b2-4590-4fc6-a9f4-3fa2a9d73b24 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:09.323075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dc16a-ebab-4282-a905-1e50573bfd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:09.323957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dc16a-ebab-4282-a905-1e50573bfd57 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:09.324208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dc16a-ebab-4282-a905-1e50573bfd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:09.324579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dc16a-ebab-4282-a905-1e50573bfd57 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:09.325410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dc16a-ebab-4282-a905-1e50573bfd57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:09.327523 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dc16a-ebab-4282-a905-1e50573bfd57 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:09.356455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1316/2610] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:09 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:09.358248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdfb63-5867-41dd-a26e-120146f78ca5 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:09.370266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdfb63-5867-41dd-a26e-120146f78ca5 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:09.381514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b58e5-bc46-4805-9692-f579da2d7c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:09.382132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b58e5-bc46-4805-9692-f579da2d7c8e admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:09.382406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b58e5-bc46-4805-9692-f579da2d7c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:09.382670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b58e5-bc46-4805-9692-f579da2d7c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:09.383345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b58e5-bc46-4805-9692-f579da2d7c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:09.385234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b58e5-bc46-4805-9692-f579da2d7c8e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:09.387159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:09.409427 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1295/2611] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:09.409920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dc16a-ebab-4282-a905-1e50573bfd57 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:09.422596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dc16a-ebab-4282-a905-1e50573bfd57 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:09.428519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79e313-d09f-42fb-8352-5ca1b27de0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:09.429542 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79e313-d09f-42fb-8352-5ca1b27de0c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:09.430792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79e313-d09f-42fb-8352-5ca1b27de0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:09.431111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79e313-d09f-42fb-8352-5ca1b27de0c0 admin admin] PATH_INFO: `/v3/projects/f7d14c8337574ca2a969d88353696db3/users/58c13f517c2442828e3799df2f2744ac/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:09.455451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79e313-d09f-42fb-8352-5ca1b27de0c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d14c8337574ca2a969d88353696db3, user_id=58c13f517c2442828e3799df2f2744ac, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:09.459112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79e313-d09f-42fb-8352-5ca1b27de0c0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:09.475447 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1317/2612] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:09 2022] PUT /identity/v3/projects/f7d14c8337574ca2a969d88353696db3/users/58c13f517c2442828e3799df2f2744ac/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:37:09.479543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b58e5-bc46-4805-9692-f579da2d7c8e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:09.491268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b58e5-bc46-4805-9692-f579da2d7c8e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:09.557393 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5da616-4ec0-4ca7-9bd0-6f911f023886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:09.558040 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da616-4ec0-4ca7-9bd0-6f911f023886 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:09.558320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da616-4ec0-4ca7-9bd0-6f911f023886 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:09.558590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5da616-4ec0-4ca7-9bd0-6f911f023886 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:09.559232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da616-4ec0-4ca7-9bd0-6f911f023886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:09.560895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5da616-4ec0-4ca7-9bd0-6f911f023886 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:09.574822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1296/2613] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:09.575975 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79e313-d09f-42fb-8352-5ca1b27de0c0 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:09.587381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79e313-d09f-42fb-8352-5ca1b27de0c0 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:09.592114 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c40ee6-f1b7-4703-bfc8-1de1914112d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:09.592809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c40ee6-f1b7-4703-bfc8-1de1914112d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:09.593090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c40ee6-f1b7-4703-bfc8-1de1914112d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:09.593364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c40ee6-f1b7-4703-bfc8-1de1914112d8 admin admin] PATH_INFO: `/v3/projects/f7d14c8337574ca2a969d88353696db3/users/58c13f517c2442828e3799df2f2744ac/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:09.595274 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c40ee6-f1b7-4703-bfc8-1de1914112d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d14c8337574ca2a969d88353696db3, user_id=58c13f517c2442828e3799df2f2744ac, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:09.597997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c40ee6-f1b7-4703-bfc8-1de1914112d8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:09.610555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1318/2614] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:09 2022] PUT /identity/v3/projects/f7d14c8337574ca2a969d88353696db3/users/58c13f517c2442828e3799df2f2744ac/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:09.621269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6a93-ae41-4b07-b72d-4a95bd6e5587 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:09.621877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6a93-ae41-4b07-b72d-4a95bd6e5587 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:09.621877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a6a93-ae41-4b07-b72d-4a95bd6e5587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:09.653045 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-f33a6a93-ae41-4b07-b72d-4a95bd6e5587 None None] MFA Rules not processed for user `58c13f517c2442828e3799df2f2744ac`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:09.654442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f33a6a93-ae41-4b07-b72d-4a95bd6e5587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:09.732233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1297/2615] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:09.744146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c40ee6-f1b7-4703-bfc8-1de1914112d8 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:09.746052 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-01c40ee6-f1b7-4703-bfc8-1de1914112d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:09.809354 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c40ee6-f1b7-4703-bfc8-1de1914112d8 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:09.813384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382f5eb-9d61-49e5-b611-bd358de0c990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:09.814243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382f5eb-9d61-49e5-b611-bd358de0c990 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:09.814694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382f5eb-9d61-49e5-b611-bd358de0c990 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:09.814964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382f5eb-9d61-49e5-b611-bd358de0c990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:09.827298 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382f5eb-9d61-49e5-b611-bd358de0c990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:09.829048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382f5eb-9d61-49e5-b611-bd358de0c990 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:09.840215 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1319/2616] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:10.373902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a6a93-ae41-4b07-b72d-4a95bd6e5587 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:10.375336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f33a6a93-ae41-4b07-b72d-4a95bd6e5587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:10.387696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a6a93-ae41-4b07-b72d-4a95bd6e5587 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:10.391191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bedce4-e5f9-4e25-a9e8-a25b5a0eb4f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:10.391793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bedce4-e5f9-4e25-a9e8-a25b5a0eb4f0 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:10.392066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bedce4-e5f9-4e25-a9e8-a25b5a0eb4f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:10.392331 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bedce4-e5f9-4e25-a9e8-a25b5a0eb4f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:10.451287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bedce4-e5f9-4e25-a9e8-a25b5a0eb4f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:10.452813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bedce4-e5f9-4e25-a9e8-a25b5a0eb4f0 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:10.462852 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1298/2617] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:10.655694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382f5eb-9d61-49e5-b611-bd358de0c990 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:10.657744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4382f5eb-9d61-49e5-b611-bd358de0c990 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:10.702998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:10.720739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382f5eb-9d61-49e5-b611-bd358de0c990 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:10.725630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810c8c1-9f29-40e8-8804-fd379025263c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:10.726248 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810c8c1-9f29-40e8-8804-fd379025263c service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:10.726526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810c8c1-9f29-40e8-8804-fd379025263c service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:10.726795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810c8c1-9f29-40e8-8804-fd379025263c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:10.728582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e810c8c1-9f29-40e8-8804-fd379025263c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:10.811433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810c8c1-9f29-40e8-8804-fd379025263c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:10.813418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810c8c1-9f29-40e8-8804-fd379025263c service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:10.825056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1320/2618] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:14.143154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bedce4-e5f9-4e25-a9e8-a25b5a0eb4f0 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:14.145362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-c3bedce4-e5f9-4e25-a9e8-a25b5a0eb4f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:14.168131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bedce4-e5f9-4e25-a9e8-a25b5a0eb4f0 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:14.172902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:14.173701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:14.173976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:14.174240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] PATH_INFO: `/v3/users/e7a36e4630904f4eac08c9de1b8235ca` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:14.175361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a36e4630904f4eac08c9de1b8235ca)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:14.178009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:14.191108 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=100 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:14.255440 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a36e4630904f4eac08c9de1b8235ca'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.256128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a36e4630904f4eac08c9de1b8235ca'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.274043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a36e4630904f4eac08c9de1b8235ca'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.294766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a36e4630904f4eac08c9de1b8235ca'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.306933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a36e4630904f4eac08c9de1b8235ca'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.317419 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1299/2619] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:14 2022] DELETE /identity/v3/users/e7a36e4630904f4eac08c9de1b8235ca => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:14.509065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810c8c1-9f29-40e8-8804-fd379025263c service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:14.522192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810c8c1-9f29-40e8-8804-fd379025263c service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:14.603597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:14.604232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:14.604531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:14.604802 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] PATH_INFO: `/v3/projects/b1bcb33a579e47518fafa960505447dc` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:14.607243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1bcb33a579e47518fafa960505447dc)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:14.609324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:14.651953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:14.654245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:14.661279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1bcb33a579e47518fafa960505447dc', 'request_id': 'req-1b75ffd3-6e77-41e4-a49a-ed285bf42236'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.667980 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1bcb33a579e47518fafa960505447dc', 'request_id': 'req-1b75ffd3-6e77-41e4-a49a-ed285bf42236'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.675695 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1bcb33a579e47518fafa960505447dc', 'request_id': 'req-1b75ffd3-6e77-41e4-a49a-ed285bf42236'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.689641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] The token cache is being invalidate because project b1bcb33a579e47518fafa960505447dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:14.689950 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.690672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-1b75ffd3-6e77-41e4-a49a-ed285bf42236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:14.693443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1321/2620] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:14 2022] DELETE /identity/v3/projects/b1bcb33a579e47518fafa960505447dc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:14.697870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2bc57-1c4e-48c5-aa46-5004cd732bd4 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:14.698151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2bc57-1c4e-48c5-aa46-5004cd732bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:14.698462 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2bc57-1c4e-48c5-aa46-5004cd732bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:14.719836 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-b5e2bc57-1c4e-48c5-aa46-5004cd732bd4 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:14.721113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b5e2bc57-1c4e-48c5-aa46-5004cd732bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:14.732128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d4575-c114-4f25-938e-d96c2bb17939 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:14.735106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:14.804204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1322/2621] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:37:14.808740 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f14fd-0784-41be-ad2a-83d4cfcc52bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:14.809361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f14fd-0784-41be-ad2a-83d4cfcc52bb service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:14.809644 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f14fd-0784-41be-ad2a-83d4cfcc52bb service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:14.809917 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f14fd-0784-41be-ad2a-83d4cfcc52bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:14.811864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-dd0f14fd-0784-41be-ad2a-83d4cfcc52bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:14.812390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d8c76-d1a2-4a42-9d67-45f80b9b6c2e None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:14.812710 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d8c76-d1a2-4a42-9d67-45f80b9b6c2e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:14.813120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d8c76-d1a2-4a42-9d67-45f80b9b6c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:14.836204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-5c8d8c76-d1a2-4a42-9d67-45f80b9b6c2e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:14.837569 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5c8d8c76-d1a2-4a42-9d67-45f80b9b6c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:14.851093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1323/2622] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:14.852007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d8c76-d1a2-4a42-9d67-45f80b9b6c2e None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:14.862597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d8c76-d1a2-4a42-9d67-45f80b9b6c2e None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:14.873276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b1bd4-a298-4454-af47-b17fea9893cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:14.873866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b1bd4-a298-4454-af47-b17fea9893cc admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:14.874144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b1bd4-a298-4454-af47-b17fea9893cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:14.874410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b1bd4-a298-4454-af47-b17fea9893cc admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:14.875207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b1bd4-a298-4454-af47-b17fea9893cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:37:14.875867 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b1bd4-a298-4454-af47-b17fea9893cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:14.877680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b1bd4-a298-4454-af47-b17fea9893cc admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:14.893983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f14fd-0784-41be-ad2a-83d4cfcc52bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:14.895188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1324/2623] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:37:14.895761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f14fd-0784-41be-ad2a-83d4cfcc52bb service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:14.899798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b1bd4-a298-4454-af47-b17fea9893cc admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:14.901421 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:14.910688 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1300/2624] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:14.911170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b1bd4-a298-4454-af47-b17fea9893cc admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:14.916012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751c16d-7b4e-4a0c-8611-8f6f1f3d8bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:14.916575 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751c16d-7b4e-4a0c-8611-8f6f1f3d8bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:14.916858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751c16d-7b4e-4a0c-8611-8f6f1f3d8bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:14.917200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751c16d-7b4e-4a0c-8611-8f6f1f3d8bd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:14.917742 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751c16d-7b4e-4a0c-8611-8f6f1f3d8bd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:14.919622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751c16d-7b4e-4a0c-8611-8f6f1f3d8bd9 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:14.948249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1325/2625] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:14 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:37:14.952786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f14fd-0784-41be-ad2a-83d4cfcc52bb service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:14.964185 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f14fd-0784-41be-ad2a-83d4cfcc52bb service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.024361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95062f-5ff7-487b-986b-d4e727f394b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.025126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95062f-5ff7-487b-986b-d4e727f394b4 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.025404 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95062f-5ff7-487b-986b-d4e727f394b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.025698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95062f-5ff7-487b-986b-d4e727f394b4 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.026489 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95062f-5ff7-487b-986b-d4e727f394b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.028592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d95062f-5ff7-487b-986b-d4e727f394b4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.051203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1301/2626] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:14 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:15.055356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751c16d-7b4e-4a0c-8611-8f6f1f3d8bd9 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.066597 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751c16d-7b4e-4a0c-8611-8f6f1f3d8bd9 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.071441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c579b-feea-4496-a72e-9bbfd22fd8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.072141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c579b-feea-4496-a72e-9bbfd22fd8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.072519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c579b-feea-4496-a72e-9bbfd22fd8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.072865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c579b-feea-4496-a72e-9bbfd22fd8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.073825 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c579b-feea-4496-a72e-9bbfd22fd8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.075492 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c579b-feea-4496-a72e-9bbfd22fd8a0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.086221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1326/2627] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:15.090328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95062f-5ff7-487b-986b-d4e727f394b4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.101316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95062f-5ff7-487b-986b-d4e727f394b4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.106159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bfe1a-c44e-4fda-906e-280844dfce65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.106756 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bfe1a-c44e-4fda-906e-280844dfce65 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.107035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bfe1a-c44e-4fda-906e-280844dfce65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.107308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bfe1a-c44e-4fda-906e-280844dfce65 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.107989 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bfe1a-c44e-4fda-906e-280844dfce65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.109907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bfe1a-c44e-4fda-906e-280844dfce65 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.121602 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1302/2628] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:37:15.126068 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c579b-feea-4496-a72e-9bbfd22fd8a0 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.138374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c579b-feea-4496-a72e-9bbfd22fd8a0 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.143899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a5771-e8d2-434a-b87a-60ce1d3ed2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.145628 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a5771-e8d2-434a-b87a-60ce1d3ed2a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.146120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a5771-e8d2-434a-b87a-60ce1d3ed2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.146578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a5771-e8d2-434a-b87a-60ce1d3ed2a8 admin admin] PATH_INFO: `/v3/projects/0f07ca922731461fbcaa0b77cd88596a/users/48eef76919bd4d25a26e8fa49c1b62e1/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.167132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a5771-e8d2-434a-b87a-60ce1d3ed2a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f07ca922731461fbcaa0b77cd88596a, user_id=48eef76919bd4d25a26e8fa49c1b62e1, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.170795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a5771-e8d2-434a-b87a-60ce1d3ed2a8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.186554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1327/2629] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:15 2022] PUT /identity/v3/projects/0f07ca922731461fbcaa0b77cd88596a/users/48eef76919bd4d25a26e8fa49c1b62e1/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:15.197979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86e235-cdc0-4532-a263-2dab5a948d6b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.198470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86e235-cdc0-4532-a263-2dab5a948d6b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.198845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86e235-cdc0-4532-a263-2dab5a948d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.227902 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-bd86e235-cdc0-4532-a263-2dab5a948d6b None None] MFA Rules not processed for user `48eef76919bd4d25a26e8fa49c1b62e1`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:15.229234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-bd86e235-cdc0-4532-a263-2dab5a948d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:15.299019 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1303/2630] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:15.304612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a5771-e8d2-434a-b87a-60ce1d3ed2a8 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.372948 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a5771-e8d2-434a-b87a-60ce1d3ed2a8 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.377688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea90bfd-0a21-4d7e-bb1d-faba2dddc49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.378319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea90bfd-0a21-4d7e-bb1d-faba2dddc49d admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.378600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea90bfd-0a21-4d7e-bb1d-faba2dddc49d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.379459 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea90bfd-0a21-4d7e-bb1d-faba2dddc49d admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.380109 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea90bfd-0a21-4d7e-bb1d-faba2dddc49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.382095 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea90bfd-0a21-4d7e-bb1d-faba2dddc49d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.410987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1328/2631] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:15 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:15.415132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86e235-cdc0-4532-a263-2dab5a948d6b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.427336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86e235-cdc0-4532-a263-2dab5a948d6b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.484437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43258f-34a7-4781-a5b3-434aed88eafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.485151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43258f-34a7-4781-a5b3-434aed88eafb admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.485433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43258f-34a7-4781-a5b3-434aed88eafb admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.485698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43258f-34a7-4781-a5b3-434aed88eafb admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.486363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43258f-34a7-4781-a5b3-434aed88eafb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.488999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43258f-34a7-4781-a5b3-434aed88eafb admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.500699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:15.512526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1304/2632] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:15 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:15.516168 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea90bfd-0a21-4d7e-bb1d-faba2dddc49d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.526639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea90bfd-0a21-4d7e-bb1d-faba2dddc49d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.531251 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e482245-96b1-4424-be29-61ad2304e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.531845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e482245-96b1-4424-be29-61ad2304e25a admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.532144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e482245-96b1-4424-be29-61ad2304e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.532460 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e482245-96b1-4424-be29-61ad2304e25a admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.533124 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e482245-96b1-4424-be29-61ad2304e25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.534713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e482245-96b1-4424-be29-61ad2304e25a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.547108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1329/2633] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:15.552439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43258f-34a7-4781-a5b3-434aed88eafb admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.565180 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43258f-34a7-4781-a5b3-434aed88eafb admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.578434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61b338-4251-44f6-b144-85ca9003a11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.579571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61b338-4251-44f6-b144-85ca9003a11d admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.579887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61b338-4251-44f6-b144-85ca9003a11d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.580245 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61b338-4251-44f6-b144-85ca9003a11d admin admin] PATH_INFO: `/v3/projects/7969edd74f7f4a778b6549b7012f1b0a/users/0bcba283da3c4ecb966f50e3b249f0b3/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.599474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61b338-4251-44f6-b144-85ca9003a11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7969edd74f7f4a778b6549b7012f1b0a, user_id=0bcba283da3c4ecb966f50e3b249f0b3, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.602201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61b338-4251-44f6-b144-85ca9003a11d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.615482 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1305/2634] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:15 2022] PUT /identity/v3/projects/7969edd74f7f4a778b6549b7012f1b0a/users/0bcba283da3c4ecb966f50e3b249f0b3/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:15.619494 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e482245-96b1-4424-be29-61ad2304e25a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.630048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e482245-96b1-4424-be29-61ad2304e25a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.641339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:15.686730 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8eccc-a5e5-4268-91b3-93ba24569c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.687254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8eccc-a5e5-4268-91b3-93ba24569c7c admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.687528 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8eccc-a5e5-4268-91b3-93ba24569c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.687795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8eccc-a5e5-4268-91b3-93ba24569c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.688470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8eccc-a5e5-4268-91b3-93ba24569c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.690571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8eccc-a5e5-4268-91b3-93ba24569c7c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.701818 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1330/2635] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:15.706325 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61b338-4251-44f6-b144-85ca9003a11d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.717856 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61b338-4251-44f6-b144-85ca9003a11d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.723535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89411f-2ce8-4d94-b67c-c708a2420577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.724227 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89411f-2ce8-4d94-b67c-c708a2420577 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.724581 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89411f-2ce8-4d94-b67c-c708a2420577 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.724860 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89411f-2ce8-4d94-b67c-c708a2420577 admin admin] PATH_INFO: `/v3/projects/7969edd74f7f4a778b6549b7012f1b0a/users/0bcba283da3c4ecb966f50e3b249f0b3/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.726911 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89411f-2ce8-4d94-b67c-c708a2420577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7969edd74f7f4a778b6549b7012f1b0a, user_id=0bcba283da3c4ecb966f50e3b249f0b3, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.729710 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89411f-2ce8-4d94-b67c-c708a2420577 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.741152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1306/2636] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:15 2022] PUT /identity/v3/projects/7969edd74f7f4a778b6549b7012f1b0a/users/0bcba283da3c4ecb966f50e3b249f0b3/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:15.752189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab0443-782a-4fff-85d1-9fb4ecb5a70a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.752698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab0443-782a-4fff-85d1-9fb4ecb5a70a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.752901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab0443-782a-4fff-85d1-9fb4ecb5a70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.782709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0fab0443-782a-4fff-85d1-9fb4ecb5a70a None None] MFA Rules not processed for user `0bcba283da3c4ecb966f50e3b249f0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:15.783898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0fab0443-782a-4fff-85d1-9fb4ecb5a70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:15.852316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1331/2637] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:15.864366 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89411f-2ce8-4d94-b67c-c708a2420577 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:15.866097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-bc89411f-2ce8-4d94-b67c-c708a2420577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:15.924920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89411f-2ce8-4d94-b67c-c708a2420577 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:15.929390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acad076-fc07-4728-b9d6-4af313b6715b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:15.930000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acad076-fc07-4728-b9d6-4af313b6715b service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:15.930280 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acad076-fc07-4728-b9d6-4af313b6715b service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:15.930553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acad076-fc07-4728-b9d6-4af313b6715b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:15.940042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acad076-fc07-4728-b9d6-4af313b6715b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:15.942061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acad076-fc07-4728-b9d6-4af313b6715b service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:15.952152 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1307/2638] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:16.029415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab0443-782a-4fff-85d1-9fb4ecb5a70a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:16.039023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab0443-782a-4fff-85d1-9fb4ecb5a70a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:16.043267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf75c16-2747-407f-a01a-3859ccd4f368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:16.043969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf75c16-2747-407f-a01a-3859ccd4f368 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:16.044249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf75c16-2747-407f-a01a-3859ccd4f368 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:16.044677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf75c16-2747-407f-a01a-3859ccd4f368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:16.108863 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf75c16-2747-407f-a01a-3859ccd4f368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:16.110321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf75c16-2747-407f-a01a-3859ccd4f368 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:16.122471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1332/2639] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:17.074727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acad076-fc07-4728-b9d6-4af313b6715b service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:17.076186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5acad076-fc07-4728-b9d6-4af313b6715b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:17.150764 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acad076-fc07-4728-b9d6-4af313b6715b service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:17.156097 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:17.157223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:17.157673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:17.158151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] PATH_INFO: `/v3/users/d01a9a05c17a49a5976ec11cc3b467fd` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:17.161233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=d01a9a05c17a49a5976ec11cc3b467fd)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:17.164157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:17.221020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd01a9a05c17a49a5976ec11cc3b467fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:17.221704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd01a9a05c17a49a5976ec11cc3b467fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:17.235782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:17.237496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd01a9a05c17a49a5976ec11cc3b467fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:17.248839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd01a9a05c17a49a5976ec11cc3b467fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:17.264224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd01a9a05c17a49a5976ec11cc3b467fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:17.274151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1308/2640] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:17 2022] DELETE /identity/v3/users/d01a9a05c17a49a5976ec11cc3b467fd => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:17.278426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179258-b7eb-4661-bc2c-a2d21b2f6443 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:17.278898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179258-b7eb-4661-bc2c-a2d21b2f6443 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:17.279001 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179258-b7eb-4661-bc2c-a2d21b2f6443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:17.320336 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-00179258-b7eb-4661-bc2c-a2d21b2f6443 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:17.321590 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-00179258-b7eb-4661-bc2c-a2d21b2f6443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:17.359050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:17.404668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1333/2641] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:17.418094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:17.421974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:17.516482 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d36c5c-69ac-42f8-abd8-0f67aa28c5ac admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:17.520728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1057314-98ad-463f-bce9-400fb75b04b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:17.521499 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1057314-98ad-463f-bce9-400fb75b04b9 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:17.521977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1057314-98ad-463f-bce9-400fb75b04b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:17.522321 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1057314-98ad-463f-bce9-400fb75b04b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:17.539574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1057314-98ad-463f-bce9-400fb75b04b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:17.545476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1057314-98ad-463f-bce9-400fb75b04b9 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:17.557899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1309/2642] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:37:17.899423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00179258-b7eb-4661-bc2c-a2d21b2f6443 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:17.901331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-00179258-b7eb-4661-bc2c-a2d21b2f6443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:17.916699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00179258-b7eb-4661-bc2c-a2d21b2f6443 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:17.921580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:17.922213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:17.922479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:17.922774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] PATH_INFO: `/v3/projects/021943111d844fe093495b2bfbc4ecdc` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:17.923889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=021943111d844fe093495b2bfbc4ecdc)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:17.927008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:17.976148 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021943111d844fe093495b2bfbc4ecdc', 'request_id': 'req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:17.982122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021943111d844fe093495b2bfbc4ecdc', 'request_id': 'req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:17.989864 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021943111d844fe093495b2bfbc4ecdc', 'request_id': 'req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.004697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] The token cache is being invalidate because project 021943111d844fe093495b2bfbc4ecdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:18.005057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.005803 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.007307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1334/2643] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:17 2022] DELETE /identity/v3/projects/021943111d844fe093495b2bfbc4ecdc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:18.011314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1057314-98ad-463f-bce9-400fb75b04b9 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:18.021523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1057314-98ad-463f-bce9-400fb75b04b9 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:18.083169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:18.083810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:18.084119 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:18.084390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] PATH_INFO: `/v3/users/75d7afe46648443bbca3869e75657190` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:18.086266 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d7afe46648443bbca3869e75657190)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:18.088268 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:18.137425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d7afe46648443bbca3869e75657190'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.138090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d7afe46648443bbca3869e75657190'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.151099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d7afe46648443bbca3869e75657190'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.157871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d7afe46648443bbca3869e75657190'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.165903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d7afe46648443bbca3869e75657190'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.171896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1310/2644] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:18 2022] DELETE /identity/v3/users/75d7afe46648443bbca3869e75657190 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:18.470373 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:18.471814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:18.490919 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6faf1-37fd-4ae5-bfbe-4d51be8d4110 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:18.552270 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:18.553012 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:18.553289 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:18.553557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] PATH_INFO: `/v3/projects/79469619f69148d99e2924112931581b` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:18.554668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79469619f69148d99e2924112931581b)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:18.556607 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:18.604741 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79469619f69148d99e2924112931581b', 'request_id': 'req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.614401 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79469619f69148d99e2924112931581b', 'request_id': 'req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.616706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:18.623209 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79469619f69148d99e2924112931581b', 'request_id': 'req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.637113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] The token cache is being invalidate because project 79469619f69148d99e2924112931581b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:18.637409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.638022 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.639501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1335/2645] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:18 2022] DELETE /identity/v3/projects/79469619f69148d99e2924112931581b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:18.643487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:18.645477 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:18.654442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:18.662443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e05685-9d07-4d61-97e3-bf48cad983cb admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:18.724582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:18.725221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:18.725502 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:18.725770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] PATH_INFO: `/v3/users/58c13f517c2442828e3799df2f2744ac` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:18.726914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c13f517c2442828e3799df2f2744ac)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:18.728899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:18.777203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c13f517c2442828e3799df2f2744ac'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.777839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c13f517c2442828e3799df2f2744ac'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.789746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c13f517c2442828e3799df2f2744ac'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.796809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c13f517c2442828e3799df2f2744ac'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.805967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d920d592-2f13-4ac1-8136-def60fa6d22c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c13f517c2442828e3799df2f2744ac'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:18.812849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1311/2646] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:18 2022] DELETE /identity/v3/users/58c13f517c2442828e3799df2f2744ac => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:19.118414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:19.119987 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:19.134100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee546c34-0047-4fbe-af5a-3a4b8dff8f36 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:19.199820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:19.200483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.200790 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.201058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] PATH_INFO: `/v3/projects/f7d14c8337574ca2a969d88353696db3` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.202160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d14c8337574ca2a969d88353696db3)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:19.204058 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:19.262989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d14c8337574ca2a969d88353696db3', 'request_id': 'req-b931e17b-a952-4cd1-a296-a4c4e4bab3db'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:19.268586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d14c8337574ca2a969d88353696db3', 'request_id': 'req-b931e17b-a952-4cd1-a296-a4c4e4bab3db'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:19.279679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d14c8337574ca2a969d88353696db3', 'request_id': 'req-b931e17b-a952-4cd1-a296-a4c4e4bab3db'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:19.293411 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] The token cache is being invalidate because project f7d14c8337574ca2a969d88353696db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:19.293716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:19.294319 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b931e17b-a952-4cd1-a296-a4c4e4bab3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:19.295798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1336/2647] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:19 2022] DELETE /identity/v3/projects/f7d14c8337574ca2a969d88353696db3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:19.302349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f2356-2041-4cb4-84ff-b5b75c5a36fd None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.302849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f2356-2041-4cb4-84ff-b5b75c5a36fd None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.302944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f2356-2041-4cb4-84ff-b5b75c5a36fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.326748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-6d8f2356-2041-4cb4-84ff-b5b75c5a36fd None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:19.327965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6d8f2356-2041-4cb4-84ff-b5b75c5a36fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:19.395374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1312/2648] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:19.405098 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21e52e-f285-433c-8f4d-b6c1b7ee74fc None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.405390 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21e52e-f285-433c-8f4d-b6c1b7ee74fc None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.405672 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21e52e-f285-433c-8f4d-b6c1b7ee74fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.425851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-cc21e52e-f285-433c-8f4d-b6c1b7ee74fc None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:19.427189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-cc21e52e-f285-433c-8f4d-b6c1b7ee74fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:19.435342 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1337/2649] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:19.439751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f2356-2041-4cb4-84ff-b5b75c5a36fd None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:19.451260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f2356-2041-4cb4-84ff-b5b75c5a36fd None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:19.456103 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e543e-d53b-4967-b151-8b9bf1269dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:19.457413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e543e-d53b-4967-b151-8b9bf1269dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.457904 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e543e-d53b-4967-b151-8b9bf1269dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.458177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e543e-d53b-4967-b151-8b9bf1269dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.458823 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e543e-d53b-4967-b151-8b9bf1269dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:37:19.459145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e543e-d53b-4967-b151-8b9bf1269dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:19.460752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e543e-d53b-4967-b151-8b9bf1269dc1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:19.472462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1313/2650] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:37:19.477084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21e52e-f285-433c-8f4d-b6c1b7ee74fc None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:19.486535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21e52e-f285-433c-8f4d-b6c1b7ee74fc None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:19.492378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984356e-bced-4c43-bda7-6238034de4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:19.493034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984356e-bced-4c43-bda7-6238034de4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.493310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984356e-bced-4c43-bda7-6238034de4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.493580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984356e-bced-4c43-bda7-6238034de4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.494221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984356e-bced-4c43-bda7-6238034de4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:19.496135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984356e-bced-4c43-bda7-6238034de4e0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:19.524264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1338/2651] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:19 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:19.528847 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e543e-d53b-4967-b151-8b9bf1269dc1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:19.539465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e543e-d53b-4967-b151-8b9bf1269dc1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:19.544029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:19.601240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42e3f2-e757-4563-850b-8fad379dbf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:19.601842 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42e3f2-e757-4563-850b-8fad379dbf72 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.602116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42e3f2-e757-4563-850b-8fad379dbf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.602383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42e3f2-e757-4563-850b-8fad379dbf72 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.602998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42e3f2-e757-4563-850b-8fad379dbf72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:19.604953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42e3f2-e757-4563-850b-8fad379dbf72 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:19.627799 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1314/2652] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:19 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:37:19.632743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984356e-bced-4c43-bda7-6238034de4e0 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:19.642999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984356e-bced-4c43-bda7-6238034de4e0 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:19.647592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aadf8f-12e0-4472-b868-2f8c79e73193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:19.648185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aadf8f-12e0-4472-b868-2f8c79e73193 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.648486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aadf8f-12e0-4472-b868-2f8c79e73193 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.648781 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aadf8f-12e0-4472-b868-2f8c79e73193 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.649481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aadf8f-12e0-4472-b868-2f8c79e73193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:19.651107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aadf8f-12e0-4472-b868-2f8c79e73193 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:19.661468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1339/2653] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:19.666157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42e3f2-e757-4563-850b-8fad379dbf72 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:19.683175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42e3f2-e757-4563-850b-8fad379dbf72 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:19.687822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2b8e9-c9bd-45a8-8bcc-3760164c11b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:19.688418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b8e9-c9bd-45a8-8bcc-3760164c11b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.688737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b8e9-c9bd-45a8-8bcc-3760164c11b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.689015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2b8e9-c9bd-45a8-8bcc-3760164c11b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.689652 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2b8e9-c9bd-45a8-8bcc-3760164c11b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:19.691889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2b8e9-c9bd-45a8-8bcc-3760164c11b0 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:19.702723 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1315/2654] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:19.707041 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aadf8f-12e0-4472-b868-2f8c79e73193 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:19.717165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aadf8f-12e0-4472-b868-2f8c79e73193 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:19.722290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4df45-193c-4a6c-bfcd-0f99dc886857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:19.723132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4df45-193c-4a6c-bfcd-0f99dc886857 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.723486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4df45-193c-4a6c-bfcd-0f99dc886857 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.723778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4df45-193c-4a6c-bfcd-0f99dc886857 admin admin] PATH_INFO: `/v3/projects/39503d1ec8de41e9a552c8de1c3c34d8/users/0dc5b806409749bc960827394b4c4d84/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.739820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4df45-193c-4a6c-bfcd-0f99dc886857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39503d1ec8de41e9a552c8de1c3c34d8, user_id=0dc5b806409749bc960827394b4c4d84, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:19.742446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4df45-193c-4a6c-bfcd-0f99dc886857 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:19.756393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1340/2655] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:19 2022] PUT /identity/v3/projects/39503d1ec8de41e9a552c8de1c3c34d8/users/0dc5b806409749bc960827394b4c4d84/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:19.767698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37be33-d413-4f2c-ba67-1aceda540966 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.767979 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37be33-d413-4f2c-ba67-1aceda540966 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.768252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37be33-d413-4f2c-ba67-1aceda540966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.796681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-7b37be33-d413-4f2c-ba67-1aceda540966 None None] MFA Rules not processed for user `0dc5b806409749bc960827394b4c4d84`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:19.797884 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7b37be33-d413-4f2c-ba67-1aceda540966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:19.865596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1316/2656] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:19.870277 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4df45-193c-4a6c-bfcd-0f99dc886857 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:19.893801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:19.940016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4df45-193c-4a6c-bfcd-0f99dc886857 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:19.945363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5e17-ac35-44d6-9a72-eac460433131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:19.945958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5e17-ac35-44d6-9a72-eac460433131 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:19.946233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5e17-ac35-44d6-9a72-eac460433131 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:19.946503 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5e17-ac35-44d6-9a72-eac460433131 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:19.947132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd5e17-ac35-44d6-9a72-eac460433131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:19.949353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd5e17-ac35-44d6-9a72-eac460433131 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:19.980446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1341/2657] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:19 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:19.981627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37be33-d413-4f2c-ba67-1aceda540966 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:19.992287 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37be33-d413-4f2c-ba67-1aceda540966 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:20.054187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199fea2-3e34-4908-91de-b6466e3cc414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:20.054816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199fea2-3e34-4908-91de-b6466e3cc414 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:20.055125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199fea2-3e34-4908-91de-b6466e3cc414 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:20.055400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199fea2-3e34-4908-91de-b6466e3cc414 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:20.056035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199fea2-3e34-4908-91de-b6466e3cc414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:20.058014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199fea2-3e34-4908-91de-b6466e3cc414 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:20.080312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1317/2658] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:19 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:37:20.085315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5e17-ac35-44d6-9a72-eac460433131 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:20.095872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5e17-ac35-44d6-9a72-eac460433131 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:20.101934 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782f682-240d-4fb3-bace-a502c9deaa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:20.102610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782f682-240d-4fb3-bace-a502c9deaa37 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:20.102978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782f682-240d-4fb3-bace-a502c9deaa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:20.103188 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782f682-240d-4fb3-bace-a502c9deaa37 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:20.103870 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782f682-240d-4fb3-bace-a502c9deaa37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:20.106184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782f682-240d-4fb3-bace-a502c9deaa37 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:20.122100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1342/2659] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:20.126239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199fea2-3e34-4908-91de-b6466e3cc414 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:20.137486 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199fea2-3e34-4908-91de-b6466e3cc414 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:20.142641 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af6675-aed7-453c-a089-c71118da128f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:20.143326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af6675-aed7-453c-a089-c71118da128f admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:20.143827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af6675-aed7-453c-a089-c71118da128f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:20.144114 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af6675-aed7-453c-a089-c71118da128f admin admin] PATH_INFO: `/v3/projects/c149c6dc33214cb68270dcf7e3970eb7/users/4bbcf2a96cc6401b810fbe0eb919b875/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:20.147140 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:20.163128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af6675-aed7-453c-a089-c71118da128f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c149c6dc33214cb68270dcf7e3970eb7, user_id=4bbcf2a96cc6401b810fbe0eb919b875, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:20.166050 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af6675-aed7-453c-a089-c71118da128f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:20.179351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1318/2660] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:20 2022] PUT /identity/v3/projects/c149c6dc33214cb68270dcf7e3970eb7/users/4bbcf2a96cc6401b810fbe0eb919b875/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:37:20.183486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782f682-240d-4fb3-bace-a502c9deaa37 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:20.194816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782f682-240d-4fb3-bace-a502c9deaa37 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:20.253930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e09ce6-44e2-4b38-a263-38d5e48c320b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:20.254550 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09ce6-44e2-4b38-a263-38d5e48c320b admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:20.254830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09ce6-44e2-4b38-a263-38d5e48c320b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:20.255099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e09ce6-44e2-4b38-a263-38d5e48c320b admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:20.255733 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e09ce6-44e2-4b38-a263-38d5e48c320b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:20.257387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e09ce6-44e2-4b38-a263-38d5e48c320b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:20.267540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1343/2661] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:37:20.272410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af6675-aed7-453c-a089-c71118da128f admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:20.283535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af6675-aed7-453c-a089-c71118da128f admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:20.289733 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d745925-b843-4ed8-876e-e037177a8156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:20.290405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d745925-b843-4ed8-876e-e037177a8156 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:20.290682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d745925-b843-4ed8-876e-e037177a8156 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:20.290950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d745925-b843-4ed8-876e-e037177a8156 admin admin] PATH_INFO: `/v3/projects/c149c6dc33214cb68270dcf7e3970eb7/users/4bbcf2a96cc6401b810fbe0eb919b875/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:20.292899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d745925-b843-4ed8-876e-e037177a8156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c149c6dc33214cb68270dcf7e3970eb7, user_id=4bbcf2a96cc6401b810fbe0eb919b875, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:20.295631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d745925-b843-4ed8-876e-e037177a8156 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:20.308367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1319/2662] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:20 2022] PUT /identity/v3/projects/c149c6dc33214cb68270dcf7e3970eb7/users/4bbcf2a96cc6401b810fbe0eb919b875/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:20.318819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c9d5e-e70c-4c2c-a327-7c47856c2174 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:20.319208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c9d5e-e70c-4c2c-a327-7c47856c2174 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:20.319602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c9d5e-e70c-4c2c-a327-7c47856c2174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:20.354000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f77c9d5e-e70c-4c2c-a327-7c47856c2174 None None] MFA Rules not processed for user `4bbcf2a96cc6401b810fbe0eb919b875`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:20.355297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f77c9d5e-e70c-4c2c-a327-7c47856c2174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:20.430260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1344/2663] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:37:20.442110 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d745925-b843-4ed8-876e-e037177a8156 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:20.443827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8d745925-b843-4ed8-876e-e037177a8156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:20.508113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d745925-b843-4ed8-876e-e037177a8156 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:20.511681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103542b-d479-46a9-8e0f-c8266c1b3036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:20.512288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103542b-d479-46a9-8e0f-c8266c1b3036 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:20.512594 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103542b-d479-46a9-8e0f-c8266c1b3036 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:20.512868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103542b-d479-46a9-8e0f-c8266c1b3036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:20.522481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103542b-d479-46a9-8e0f-c8266c1b3036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:20.523913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103542b-d479-46a9-8e0f-c8266c1b3036 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:20.533147 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1320/2664] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:20.611337 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c9d5e-e70c-4c2c-a327-7c47856c2174 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:20.621358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c9d5e-e70c-4c2c-a327-7c47856c2174 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:20.624970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639d20d-b57e-465e-b55e-d2591188b460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:20.625600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639d20d-b57e-465e-b55e-d2591188b460 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:20.625911 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639d20d-b57e-465e-b55e-d2591188b460 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:20.626179 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639d20d-b57e-465e-b55e-d2591188b460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:20.635066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:20.686474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639d20d-b57e-465e-b55e-d2591188b460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:20.687925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639d20d-b57e-465e-b55e-d2591188b460 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:20.697726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1345/2665] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:21.002116 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103542b-d479-46a9-8e0f-c8266c1b3036 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:21.069455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103542b-d479-46a9-8e0f-c8266c1b3036 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:21.074211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:21.074863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:21.075141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:21.075409 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] PATH_INFO: `/v3/users/0dc5b806409749bc960827394b4c4d84` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:21.076567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dc5b806409749bc960827394b4c4d84)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:21.078483 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:21.127141 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dc5b806409749bc960827394b4c4d84'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.127805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dc5b806409749bc960827394b4c4d84'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.141852 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dc5b806409749bc960827394b4c4d84'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.149524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dc5b806409749bc960827394b4c4d84'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.151466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:21.158504 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dc5b806409749bc960827394b4c4d84'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.166240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1321/2666] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:21 2022] DELETE /identity/v3/users/0dc5b806409749bc960827394b4c4d84 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:21.170752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d9c4f-7e67-4115-bee6-c283c9f1db33 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:21.171034 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d9c4f-7e67-4115-bee6-c283c9f1db33 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:21.171306 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d9c4f-7e67-4115-bee6-c283c9f1db33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:21.191433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-4b2d9c4f-7e67-4115-bee6-c283c9f1db33 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:21.193358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4b2d9c4f-7e67-4115-bee6-c283c9f1db33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:21.255624 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1346/2667] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:21.265320 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:21.266742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:21.326347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7e773-8f26-44ba-9460-1dc6ce63e381 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:21.330016 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26cee7-4d37-48ec-ba5d-153fa11be11a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:21.330621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26cee7-4d37-48ec-ba5d-153fa11be11a service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:21.330896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26cee7-4d37-48ec-ba5d-153fa11be11a service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:21.331167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26cee7-4d37-48ec-ba5d-153fa11be11a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:21.340588 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26cee7-4d37-48ec-ba5d-153fa11be11a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:21.341992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26cee7-4d37-48ec-ba5d-153fa11be11a service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:21.351834 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1322/2668] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:21.630984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d9c4f-7e67-4115-bee6-c283c9f1db33 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:21.632689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4b2d9c4f-7e67-4115-bee6-c283c9f1db33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:21.645055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d9c4f-7e67-4115-bee6-c283c9f1db33 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:21.655840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:21.655840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:21.655840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:21.655840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] PATH_INFO: `/v3/projects/39503d1ec8de41e9a552c8de1c3c34d8` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:21.657049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] RBAC: Authorizing `identity:delete_project(project_id=39503d1ec8de41e9a552c8de1c3c34d8)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:21.657049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:21.708900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39503d1ec8de41e9a552c8de1c3c34d8', 'request_id': 'req-622a53e6-aaca-4f82-9945-587b3814e53a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.715521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39503d1ec8de41e9a552c8de1c3c34d8', 'request_id': 'req-622a53e6-aaca-4f82-9945-587b3814e53a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.722890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39503d1ec8de41e9a552c8de1c3c34d8', 'request_id': 'req-622a53e6-aaca-4f82-9945-587b3814e53a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.736021 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] The token cache is being invalidate because project 39503d1ec8de41e9a552c8de1c3c34d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:21.736305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.736945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.739151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1347/2669] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:21 2022] DELETE /identity/v3/projects/39503d1ec8de41e9a552c8de1c3c34d8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:21.743199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26cee7-4d37-48ec-ba5d-153fa11be11a service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:21.752961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26cee7-4d37-48ec-ba5d-153fa11be11a service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:21.813432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:21.814068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:21.814369 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:21.814638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] PATH_INFO: `/v3/users/4bbcf2a96cc6401b810fbe0eb919b875` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:21.815780 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bbcf2a96cc6401b810fbe0eb919b875)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:21.818418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:21.872531 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bbcf2a96cc6401b810fbe0eb919b875'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.873263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bbcf2a96cc6401b810fbe0eb919b875'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.889384 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bbcf2a96cc6401b810fbe0eb919b875'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.897131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bbcf2a96cc6401b810fbe0eb919b875'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.906715 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dbe0bce8-b350-45de-a0c7-f29809f66078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bbcf2a96cc6401b810fbe0eb919b875'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:21.913350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1323/2670] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:21 2022] DELETE /identity/v3/users/4bbcf2a96cc6401b810fbe0eb919b875 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:22.212156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:22.213983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:22.231855 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a53e6-aaca-4f82-9945-587b3814e53a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:22.272693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:22.300906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:22.301540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.301813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.302085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] PATH_INFO: `/v3/projects/c149c6dc33214cb68270dcf7e3970eb7` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.303176 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c149c6dc33214cb68270dcf7e3970eb7)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:22.305108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:22.361343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c149c6dc33214cb68270dcf7e3970eb7', 'request_id': 'req-2ec44963-9cb6-4f13-be15-7679a91f5e57'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:22.367509 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c149c6dc33214cb68270dcf7e3970eb7', 'request_id': 'req-2ec44963-9cb6-4f13-be15-7679a91f5e57'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:22.373671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c149c6dc33214cb68270dcf7e3970eb7', 'request_id': 'req-2ec44963-9cb6-4f13-be15-7679a91f5e57'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:22.386103 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] The token cache is being invalidate because project c149c6dc33214cb68270dcf7e3970eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:22.386315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:22.386945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-2ec44963-9cb6-4f13-be15-7679a91f5e57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:22.388465 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1348/2671] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:22 2022] DELETE /identity/v3/projects/c149c6dc33214cb68270dcf7e3970eb7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:22.393603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f9a5c-ffd7-4165-997e-242dc8e49e5f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.394124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f9a5c-ffd7-4165-997e-242dc8e49e5f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.394124 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f9a5c-ffd7-4165-997e-242dc8e49e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.415102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-fd3f9a5c-ffd7-4165-997e-242dc8e49e5f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:22.416237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fd3f9a5c-ffd7-4165-997e-242dc8e49e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:22.425361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:22.481565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1324/2672] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:22.491272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bff310-398e-49e1-a560-7c82da8eff80 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.491586 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bff310-398e-49e1-a560-7c82da8eff80 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.491891 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bff310-398e-49e1-a560-7c82da8eff80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.512845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-73bff310-398e-49e1-a560-7c82da8eff80 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:22.514299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-73bff310-398e-49e1-a560-7c82da8eff80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:22.524430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1349/2673] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:22.528852 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f9a5c-ffd7-4165-997e-242dc8e49e5f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:22.539495 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f9a5c-ffd7-4165-997e-242dc8e49e5f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:22.544084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664b9c4-2162-47cf-a1e6-8ee0251a0501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:22.544737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664b9c4-2162-47cf-a1e6-8ee0251a0501 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.545020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664b9c4-2162-47cf-a1e6-8ee0251a0501 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.545294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664b9c4-2162-47cf-a1e6-8ee0251a0501 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.545949 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664b9c4-2162-47cf-a1e6-8ee0251a0501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:37:22.546251 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664b9c4-2162-47cf-a1e6-8ee0251a0501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:22.547862 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664b9c4-2162-47cf-a1e6-8ee0251a0501 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:22.559953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1325/2674] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:37:22.564458 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bff310-398e-49e1-a560-7c82da8eff80 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:22.575066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bff310-398e-49e1-a560-7c82da8eff80 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:22.579698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a080278-130b-4669-ba2b-cd2545697e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:22.580295 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a080278-130b-4669-ba2b-cd2545697e22 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.580603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a080278-130b-4669-ba2b-cd2545697e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.580875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a080278-130b-4669-ba2b-cd2545697e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.582048 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a080278-130b-4669-ba2b-cd2545697e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:22.583945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a080278-130b-4669-ba2b-cd2545697e22 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:22.612583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1350/2675] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:22 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:22.616869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664b9c4-2162-47cf-a1e6-8ee0251a0501 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:22.627178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664b9c4-2162-47cf-a1e6-8ee0251a0501 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:22.684792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728b901-d6d1-4e38-b8df-22a784bd2819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:22.685362 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b901-d6d1-4e38-b8df-22a784bd2819 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.685638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b901-d6d1-4e38-b8df-22a784bd2819 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.685906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b901-d6d1-4e38-b8df-22a784bd2819 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.686671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728b901-d6d1-4e38-b8df-22a784bd2819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:22.688742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728b901-d6d1-4e38-b8df-22a784bd2819 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:22.712084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1326/2676] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:22 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:37:22.716268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a080278-130b-4669-ba2b-cd2545697e22 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:22.727310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a080278-130b-4669-ba2b-cd2545697e22 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:22.740264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea95d-5e8d-401b-90df-429ba9c059ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:22.740920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea95d-5e8d-401b-90df-429ba9c059ae admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.741200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea95d-5e8d-401b-90df-429ba9c059ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.741466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea95d-5e8d-401b-90df-429ba9c059ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.742152 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ea95d-5e8d-401b-90df-429ba9c059ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:22.743797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ea95d-5e8d-401b-90df-429ba9c059ae admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:22.757101 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1351/2677] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:22.761104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728b901-d6d1-4e38-b8df-22a784bd2819 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:22.771779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728b901-d6d1-4e38-b8df-22a784bd2819 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:22.776402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcf23b-b3dd-4c87-99fd-4b0f39142eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:22.777025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcf23b-b3dd-4c87-99fd-4b0f39142eec admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.777304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcf23b-b3dd-4c87-99fd-4b0f39142eec admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.777576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcf23b-b3dd-4c87-99fd-4b0f39142eec admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.778213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcf23b-b3dd-4c87-99fd-4b0f39142eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:22.779812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcf23b-b3dd-4c87-99fd-4b0f39142eec admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:22.792562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1327/2678] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:37:22.796682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea95d-5e8d-401b-90df-429ba9c059ae admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:22.807925 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea95d-5e8d-401b-90df-429ba9c059ae admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:22.812696 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03427848-39af-43fb-b5f0-52596b393635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:22.813348 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03427848-39af-43fb-b5f0-52596b393635 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.813627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03427848-39af-43fb-b5f0-52596b393635 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.813900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03427848-39af-43fb-b5f0-52596b393635 admin admin] PATH_INFO: `/v3/projects/5cc350c92103421b8151d3ac7b37f64a/users/5924271d811b4732b605a664d113aeb4/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.830900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03427848-39af-43fb-b5f0-52596b393635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc350c92103421b8151d3ac7b37f64a, user_id=5924271d811b4732b605a664d113aeb4, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:22.834185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03427848-39af-43fb-b5f0-52596b393635 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:22.848128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1352/2679] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:22 2022] PUT /identity/v3/projects/5cc350c92103421b8151d3ac7b37f64a/users/5924271d811b4732b605a664d113aeb4/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:22.858683 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03cdfa-9bcf-45ed-8047-e178f523ca6e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:22.858970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03cdfa-9bcf-45ed-8047-e178f523ca6e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:22.859269 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03cdfa-9bcf-45ed-8047-e178f523ca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:22.891630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-0a03cdfa-9bcf-45ed-8047-e178f523ca6e None None] MFA Rules not processed for user `5924271d811b4732b605a664d113aeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:22.892963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0a03cdfa-9bcf-45ed-8047-e178f523ca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:22.962158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1328/2680] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:22.966614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03427848-39af-43fb-b5f0-52596b393635 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.031444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03427848-39af-43fb-b5f0-52596b393635 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.036719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a6f1e-4b0a-440b-ac57-45cd016708d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.037328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a6f1e-4b0a-440b-ac57-45cd016708d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.037608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a6f1e-4b0a-440b-ac57-45cd016708d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.037878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a6f1e-4b0a-440b-ac57-45cd016708d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.038504 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a6f1e-4b0a-440b-ac57-45cd016708d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.040408 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a6f1e-4b0a-440b-ac57-45cd016708d2 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.045057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:23.069315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1353/2681] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:22 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:23.074350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03cdfa-9bcf-45ed-8047-e178f523ca6e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.085762 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03cdfa-9bcf-45ed-8047-e178f523ca6e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.090479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:23.149316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1d432-cf42-4a05-971e-879afeb98ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.150020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1d432-cf42-4a05-971e-879afeb98ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.150253 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1d432-cf42-4a05-971e-879afeb98ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.150537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1d432-cf42-4a05-971e-879afeb98ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.151238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1d432-cf42-4a05-971e-879afeb98ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.153302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1d432-cf42-4a05-971e-879afeb98ac4 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.179747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1329/2682] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:23 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:23.183677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a6f1e-4b0a-440b-ac57-45cd016708d2 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.194418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a6f1e-4b0a-440b-ac57-45cd016708d2 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.199409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584b329-dd2c-4235-870e-10bed3e86951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.200011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584b329-dd2c-4235-870e-10bed3e86951 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.200286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584b329-dd2c-4235-870e-10bed3e86951 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.200598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584b329-dd2c-4235-870e-10bed3e86951 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.201241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584b329-dd2c-4235-870e-10bed3e86951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.202841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584b329-dd2c-4235-870e-10bed3e86951 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.217386 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1354/2683] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:23.223128 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1d432-cf42-4a05-971e-879afeb98ac4 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.236865 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1d432-cf42-4a05-971e-879afeb98ac4 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.242566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7e08c-246c-40c8-80c3-51251853251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.243175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7e08c-246c-40c8-80c3-51251853251a admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.243456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7e08c-246c-40c8-80c3-51251853251a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.243729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7e08c-246c-40c8-80c3-51251853251a admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.244380 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7e08c-246c-40c8-80c3-51251853251a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.252879 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7e08c-246c-40c8-80c3-51251853251a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.260470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1330/2684] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:23.262671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584b329-dd2c-4235-870e-10bed3e86951 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.273301 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584b329-dd2c-4235-870e-10bed3e86951 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.282540 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891e059-2266-4faa-bad5-9878c8bbb589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.283220 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891e059-2266-4faa-bad5-9878c8bbb589 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.283500 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891e059-2266-4faa-bad5-9878c8bbb589 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.283775 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891e059-2266-4faa-bad5-9878c8bbb589 admin admin] PATH_INFO: `/v3/projects/5495e8d78a2f4029b2fd444bf766348b/users/7afc0455b1ca413eae0edf6b87d2e5af/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.300107 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891e059-2266-4faa-bad5-9878c8bbb589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5495e8d78a2f4029b2fd444bf766348b, user_id=7afc0455b1ca413eae0edf6b87d2e5af, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.303053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891e059-2266-4faa-bad5-9878c8bbb589 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.319392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1355/2685] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:23 2022] PUT /identity/v3/projects/5495e8d78a2f4029b2fd444bf766348b/users/7afc0455b1ca413eae0edf6b87d2e5af/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:23.331334 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ca2ea-ec80-4964-b092-49073c3abf6c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.332475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ca2ea-ec80-4964-b092-49073c3abf6c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.332782 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ca2ea-ec80-4964-b092-49073c3abf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.361859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-1b2ca2ea-ec80-4964-b092-49073c3abf6c None None] MFA Rules not processed for user `7afc0455b1ca413eae0edf6b87d2e5af`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:23.363092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-1b2ca2ea-ec80-4964-b092-49073c3abf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:23.433809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1331/2686] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:23.438440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891e059-2266-4faa-bad5-9878c8bbb589 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.508552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891e059-2266-4faa-bad5-9878c8bbb589 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.513384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43a29a-31b1-4bb8-bb34-e41870ca6aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.513990 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43a29a-31b1-4bb8-bb34-e41870ca6aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.514268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43a29a-31b1-4bb8-bb34-e41870ca6aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.514547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43a29a-31b1-4bb8-bb34-e41870ca6aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.515181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43a29a-31b1-4bb8-bb34-e41870ca6aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.517138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43a29a-31b1-4bb8-bb34-e41870ca6aa3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.552181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1356/2687] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:23 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:23.553868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ca2ea-ec80-4964-b092-49073c3abf6c None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.564943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ca2ea-ec80-4964-b092-49073c3abf6c None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.632028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d8799-5230-476c-8bfd-88f2e676539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.632870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d8799-5230-476c-8bfd-88f2e676539f admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.633219 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d8799-5230-476c-8bfd-88f2e676539f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.633561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d8799-5230-476c-8bfd-88f2e676539f admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.634228 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d8799-5230-476c-8bfd-88f2e676539f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.636374 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d8799-5230-476c-8bfd-88f2e676539f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.658832 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1332/2688] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:23 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:37:23.663480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43a29a-31b1-4bb8-bb34-e41870ca6aa3 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.674425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43a29a-31b1-4bb8-bb34-e41870ca6aa3 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.679440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b741158-8eb1-40ef-868f-ae6f4a34d86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.680053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b741158-8eb1-40ef-868f-ae6f4a34d86f admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.680333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b741158-8eb1-40ef-868f-ae6f4a34d86f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.680664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b741158-8eb1-40ef-868f-ae6f4a34d86f admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.681300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b741158-8eb1-40ef-868f-ae6f4a34d86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.682921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b741158-8eb1-40ef-868f-ae6f4a34d86f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.701436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1357/2689] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:23.705709 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d8799-5230-476c-8bfd-88f2e676539f admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.717628 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d8799-5230-476c-8bfd-88f2e676539f admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.722694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c62181-dac7-4717-bb20-a05228ef0fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.723399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c62181-dac7-4717-bb20-a05228ef0fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.723679 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c62181-dac7-4717-bb20-a05228ef0fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.723952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c62181-dac7-4717-bb20-a05228ef0fcf admin admin] PATH_INFO: `/v3/projects/9e7b67fba0e4404dac22dcff1ff8ec7b/users/177a6b9aa74d496cb1ddca312f55237a/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.726837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:23.742841 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c62181-dac7-4717-bb20-a05228ef0fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7b67fba0e4404dac22dcff1ff8ec7b, user_id=177a6b9aa74d496cb1ddca312f55237a, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.745806 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c62181-dac7-4717-bb20-a05228ef0fcf admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.761095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1333/2690] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:23 2022] PUT /identity/v3/projects/9e7b67fba0e4404dac22dcff1ff8ec7b/users/177a6b9aa74d496cb1ddca312f55237a/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:23.765202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b741158-8eb1-40ef-868f-ae6f4a34d86f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.776568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b741158-8eb1-40ef-868f-ae6f4a34d86f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.841993 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1326cb-364b-41be-b476-f1e07c197312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.842606 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1326cb-364b-41be-b476-f1e07c197312 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.842881 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1326cb-364b-41be-b476-f1e07c197312 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.843150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1326cb-364b-41be-b476-f1e07c197312 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.844119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1326cb-364b-41be-b476-f1e07c197312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.846283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1326cb-364b-41be-b476-f1e07c197312 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.857746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1358/2691] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:23.862202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c62181-dac7-4717-bb20-a05228ef0fcf admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:23.874532 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c62181-dac7-4717-bb20-a05228ef0fcf admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:23.879882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8322c07-010a-426a-a873-b398e2dc7aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:23.880741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8322c07-010a-426a-a873-b398e2dc7aea admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.881021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8322c07-010a-426a-a873-b398e2dc7aea admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.881296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8322c07-010a-426a-a873-b398e2dc7aea admin admin] PATH_INFO: `/v3/projects/9e7b67fba0e4404dac22dcff1ff8ec7b/users/177a6b9aa74d496cb1ddca312f55237a/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.883454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8322c07-010a-426a-a873-b398e2dc7aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7b67fba0e4404dac22dcff1ff8ec7b, user_id=177a6b9aa74d496cb1ddca312f55237a, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:23.886316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8322c07-010a-426a-a873-b398e2dc7aea admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:23.897783 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1334/2692] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:23 2022] PUT /identity/v3/projects/9e7b67fba0e4404dac22dcff1ff8ec7b/users/177a6b9aa74d496cb1ddca312f55237a/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:23.907969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c418e3-9508-43f0-a069-6a5e6162c9e2 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:23.908250 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c418e3-9508-43f0-a069-6a5e6162c9e2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:23.908547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c418e3-9508-43f0-a069-6a5e6162c9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:23.939071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-49c418e3-9508-43f0-a069-6a5e6162c9e2 None None] MFA Rules not processed for user `177a6b9aa74d496cb1ddca312f55237a`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:23.940293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-49c418e3-9508-43f0-a069-6a5e6162c9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:23.949182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:24.020369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1359/2693] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:24.034092 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8322c07-010a-426a-a873-b398e2dc7aea admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:24.036432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b8322c07-010a-426a-a873-b398e2dc7aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:24.097829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8322c07-010a-426a-a873-b398e2dc7aea admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:24.101772 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c45b6b-1813-4505-bb68-9cb4189b6e62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:24.102448 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c45b6b-1813-4505-bb68-9cb4189b6e62 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:24.102749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c45b6b-1813-4505-bb68-9cb4189b6e62 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:24.103022 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c45b6b-1813-4505-bb68-9cb4189b6e62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:24.162312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c45b6b-1813-4505-bb68-9cb4189b6e62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:24.163849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c45b6b-1813-4505-bb68-9cb4189b6e62 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:24.173722 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1335/2694] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:25.703520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c418e3-9508-43f0-a069-6a5e6162c9e2 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:25.714079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c418e3-9508-43f0-a069-6a5e6162c9e2 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:25.717629 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef69369-2a7c-49dc-bd10-0b1c141bed81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:25.718247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef69369-2a7c-49dc-bd10-0b1c141bed81 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:25.718521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef69369-2a7c-49dc-bd10-0b1c141bed81 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:25.718788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef69369-2a7c-49dc-bd10-0b1c141bed81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:25.778505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef69369-2a7c-49dc-bd10-0b1c141bed81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:25.780088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef69369-2a7c-49dc-bd10-0b1c141bed81 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:25.790362 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1360/2695] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:26.362420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c45b6b-1813-4505-bb68-9cb4189b6e62 service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:26.372031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c45b6b-1813-4505-bb68-9cb4189b6e62 service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:26.376076 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5945a-ea30-4922-841a-4e01482fa2d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:26.376870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5945a-ea30-4922-841a-4e01482fa2d5 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:26.377235 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5945a-ea30-4922-841a-4e01482fa2d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:26.377660 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5945a-ea30-4922-841a-4e01482fa2d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:26.392283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5945a-ea30-4922-841a-4e01482fa2d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:26.393905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5945a-ea30-4922-841a-4e01482fa2d5 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:26.403936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1336/2696] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:30.346995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef69369-2a7c-49dc-bd10-0b1c141bed81 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:30.417605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef69369-2a7c-49dc-bd10-0b1c141bed81 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:30.422395 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:30.423130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:30.423446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:30.423750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] PATH_INFO: `/v3/users/5924271d811b4732b605a664d113aeb4` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:30.425163 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5924271d811b4732b605a664d113aeb4)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:30.427942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:30.480748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:30.482884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5924271d811b4732b605a664d113aeb4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:30.483011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5924271d811b4732b605a664d113aeb4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:30.492453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5924271d811b4732b605a664d113aeb4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:30.503749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5924271d811b4732b605a664d113aeb4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:30.505290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5924271d811b4732b605a664d113aeb4'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:30.540110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1361/2697] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:30 2022] DELETE /identity/v3/users/5924271d811b4732b605a664d113aeb4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:30.544429 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8f862-6a31-4995-90ff-b0d00d1f591b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:30.544994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8f862-6a31-4995-90ff-b0d00d1f591b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:30.544994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8f862-6a31-4995-90ff-b0d00d1f591b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:30.568991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ccb8f862-6a31-4995-90ff-b0d00d1f591b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:30.570248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ccb8f862-6a31-4995-90ff-b0d00d1f591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:30.643585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1337/2698] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:30.654000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:30.655437 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:30.713746 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485748a-e3dc-404e-84b6-7cac5095a3de admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:30.717346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb59e1-5916-424b-955b-34f545eb9a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:30.717939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb59e1-5916-424b-955b-34f545eb9a8a service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:30.718214 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb59e1-5916-424b-955b-34f545eb9a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:30.718486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb59e1-5916-424b-955b-34f545eb9a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:30.728231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb59e1-5916-424b-955b-34f545eb9a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:30.729777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb59e1-5916-424b-955b-34f545eb9a8a service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:30.739878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1362/2699] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:31.029590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8f862-6a31-4995-90ff-b0d00d1f591b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:31.031252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ccb8f862-6a31-4995-90ff-b0d00d1f591b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:31.044770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8f862-6a31-4995-90ff-b0d00d1f591b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:31.049619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:31.050249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:31.050526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:31.050801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] PATH_INFO: `/v3/projects/5cc350c92103421b8151d3ac7b37f64a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:31.051968 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc350c92103421b8151d3ac7b37f64a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:31.053926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:31.060896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:31.110612 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc350c92103421b8151d3ac7b37f64a', 'request_id': 'req-8f3be992-d839-4042-a43a-929f8b18bca1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.116200 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc350c92103421b8151d3ac7b37f64a', 'request_id': 'req-8f3be992-d839-4042-a43a-929f8b18bca1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.122680 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc350c92103421b8151d3ac7b37f64a', 'request_id': 'req-8f3be992-d839-4042-a43a-929f8b18bca1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.138673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] The token cache is being invalidate because project 5cc350c92103421b8151d3ac7b37f64a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:31.139049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.139674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.141244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1338/2700] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:31 2022] DELETE /identity/v3/projects/5cc350c92103421b8151d3ac7b37f64a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:31.145423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb59e1-5916-424b-955b-34f545eb9a8a service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:31.155100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb59e1-5916-424b-955b-34f545eb9a8a service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:31.211965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:31.212653 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:31.212962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:31.213233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] PATH_INFO: `/v3/users/7afc0455b1ca413eae0edf6b87d2e5af` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:31.214363 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7afc0455b1ca413eae0edf6b87d2e5af)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:31.216402 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:31.263858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7afc0455b1ca413eae0edf6b87d2e5af'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.264577 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7afc0455b1ca413eae0edf6b87d2e5af'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.272092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7afc0455b1ca413eae0edf6b87d2e5af'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.281546 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7afc0455b1ca413eae0edf6b87d2e5af'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.282183 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7afc0455b1ca413eae0edf6b87d2e5af'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.287328 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:31.303074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1363/2701] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:31 2022] DELETE /identity/v3/users/7afc0455b1ca413eae0edf6b87d2e5af => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:31.573789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:31.575254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:31.589044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3be992-d839-4042-a43a-929f8b18bca1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:31.678778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:31.679413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:31.679694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:31.679963 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] PATH_INFO: `/v3/projects/5495e8d78a2f4029b2fd444bf766348b` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:31.681122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5495e8d78a2f4029b2fd444bf766348b)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:31.683038 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:31.732094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5495e8d78a2f4029b2fd444bf766348b', 'request_id': 'req-bb386e40-d570-4fec-a988-92f2ff58f1aa'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.737698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5495e8d78a2f4029b2fd444bf766348b', 'request_id': 'req-bb386e40-d570-4fec-a988-92f2ff58f1aa'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.744187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5495e8d78a2f4029b2fd444bf766348b', 'request_id': 'req-bb386e40-d570-4fec-a988-92f2ff58f1aa'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.759831 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] The token cache is being invalidate because project 5495e8d78a2f4029b2fd444bf766348b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:31.760294 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.761136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.762913 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1339/2702] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:31 2022] DELETE /identity/v3/projects/5495e8d78a2f4029b2fd444bf766348b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:31.767016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:31.768430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:31.781466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9802a-d478-4986-be7d-649604c55442 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:31.841817 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:31.842449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:31.842743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:31.843019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] PATH_INFO: `/v3/users/177a6b9aa74d496cb1ddca312f55237a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:31.844135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] RBAC: Authorizing `identity:delete_user(user_id=177a6b9aa74d496cb1ddca312f55237a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:31.846083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:31.899389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '177a6b9aa74d496cb1ddca312f55237a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.900044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '177a6b9aa74d496cb1ddca312f55237a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.907572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '177a6b9aa74d496cb1ddca312f55237a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.916147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '177a6b9aa74d496cb1ddca312f55237a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.916827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-47ea0f9a-36c4-4a53-af57-e606f308902a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '177a6b9aa74d496cb1ddca312f55237a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:31.950740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1364/2703] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:37:31 2022] DELETE /identity/v3/users/177a6b9aa74d496cb1ddca312f55237a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:32.237634 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:32.239877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:32.255089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386e40-d570-4fec-a988-92f2ff58f1aa admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:32.280465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:32.319028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:32.319658 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.319933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.320202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] PATH_INFO: `/v3/projects/9e7b67fba0e4404dac22dcff1ff8ec7b` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.321345 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7b67fba0e4404dac22dcff1ff8ec7b)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:32.323513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:32.369838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7b67fba0e4404dac22dcff1ff8ec7b', 'request_id': 'req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:32.375338 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7b67fba0e4404dac22dcff1ff8ec7b', 'request_id': 'req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:32.382333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7b67fba0e4404dac22dcff1ff8ec7b', 'request_id': 'req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:32.395540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] The token cache is being invalidate because project 9e7b67fba0e4404dac22dcff1ff8ec7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:37:32.395824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:32.396454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0ceff5e2-a5b1-433c-b7fc-a4fe2e7be193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:37:32.397961 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1340/2704] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:37:32 2022] DELETE /identity/v3/projects/9e7b67fba0e4404dac22dcff1ff8ec7b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:37:32.403309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f53564-41b4-4aa6-b4b6-e819313409bf None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.403534 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f53564-41b4-4aa6-b4b6-e819313409bf None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.403798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f53564-41b4-4aa6-b4b6-e819313409bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.423522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-55f53564-41b4-4aa6-b4b6-e819313409bf None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:32.424774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-55f53564-41b4-4aa6-b4b6-e819313409bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:32.483132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1365/2705] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:32.491142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2db1f3-14da-4406-9469-a99319e55239 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.491536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2db1f3-14da-4406-9469-a99319e55239 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.491837 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2db1f3-14da-4406-9469-a99319e55239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.511351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-dd2db1f3-14da-4406-9469-a99319e55239 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:32.512597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-dd2db1f3-14da-4406-9469-a99319e55239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:32.521214 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1341/2706] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:32.525826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f53564-41b4-4aa6-b4b6-e819313409bf None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:32.532644 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:32.539504 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f53564-41b4-4aa6-b4b6-e819313409bf None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:32.545339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48319-bbe0-4930-b8a1-0d9f5bbb66d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:32.545930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48319-bbe0-4930-b8a1-0d9f5bbb66d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.546208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48319-bbe0-4930-b8a1-0d9f5bbb66d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.546475 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a48319-bbe0-4930-b8a1-0d9f5bbb66d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.547144 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48319-bbe0-4930-b8a1-0d9f5bbb66d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:37:32.547441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48319-bbe0-4930-b8a1-0d9f5bbb66d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:32.549104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a48319-bbe0-4930-b8a1-0d9f5bbb66d5 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:32.560860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1366/2707] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:37:32.565476 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2db1f3-14da-4406-9469-a99319e55239 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:32.575420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2db1f3-14da-4406-9469-a99319e55239 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:32.580064 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58139f4-f04a-43e2-ac78-5a07bf15ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:32.580689 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58139f4-f04a-43e2-ac78-5a07bf15ddeb admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.580969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58139f4-f04a-43e2-ac78-5a07bf15ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.581236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58139f4-f04a-43e2-ac78-5a07bf15ddeb admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.581857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58139f4-f04a-43e2-ac78-5a07bf15ddeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:32.584305 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58139f4-f04a-43e2-ac78-5a07bf15ddeb admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:32.610872 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1342/2708] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:37:32 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:32.615158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48319-bbe0-4930-b8a1-0d9f5bbb66d5 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:32.625196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a48319-bbe0-4930-b8a1-0d9f5bbb66d5 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:32.681279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9a1cf-6bd5-440d-947e-71d21cd6ad71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:32.681898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9a1cf-6bd5-440d-947e-71d21cd6ad71 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.682174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9a1cf-6bd5-440d-947e-71d21cd6ad71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.682445 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9a1cf-6bd5-440d-947e-71d21cd6ad71 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.683056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9a1cf-6bd5-440d-947e-71d21cd6ad71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:32.684985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c9a1cf-6bd5-440d-947e-71d21cd6ad71 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:32.711463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1367/2709] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:37:32 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:37:32.715601 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58139f4-f04a-43e2-ac78-5a07bf15ddeb admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:32.729395 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58139f4-f04a-43e2-ac78-5a07bf15ddeb admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:32.735087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e76d56-e907-46a2-9df8-bacf792e91f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:32.735686 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e76d56-e907-46a2-9df8-bacf792e91f7 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.735962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e76d56-e907-46a2-9df8-bacf792e91f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.736232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e76d56-e907-46a2-9df8-bacf792e91f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.736908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e76d56-e907-46a2-9df8-bacf792e91f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:32.738526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e76d56-e907-46a2-9df8-bacf792e91f7 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:32.748567 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1343/2710] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:32.752474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9a1cf-6bd5-440d-947e-71d21cd6ad71 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:32.763423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9a1cf-6bd5-440d-947e-71d21cd6ad71 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:32.768115 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bbae4-156b-4c53-ab4e-310d95214587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:32.768819 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bbae4-156b-4c53-ab4e-310d95214587 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.769160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bbae4-156b-4c53-ab4e-310d95214587 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.769409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bbae4-156b-4c53-ab4e-310d95214587 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.770780 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bbae4-156b-4c53-ab4e-310d95214587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:32.772563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bbae4-156b-4c53-ab4e-310d95214587 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:32.783193 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1368/2711] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:37:32.787452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e76d56-e907-46a2-9df8-bacf792e91f7 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:32.797541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e76d56-e907-46a2-9df8-bacf792e91f7 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:32.802406 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e033f3b-eae3-4ca4-a864-55d89057f506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:32.803079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e033f3b-eae3-4ca4-a864-55d89057f506 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.803439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e033f3b-eae3-4ca4-a864-55d89057f506 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.803627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e033f3b-eae3-4ca4-a864-55d89057f506 admin admin] PATH_INFO: `/v3/projects/2cba9b74bb6b4316a270c1bd7e0a59b6/users/5b9c9f9b19974d6fa9abbe87651fac9d/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.818760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e033f3b-eae3-4ca4-a864-55d89057f506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cba9b74bb6b4316a270c1bd7e0a59b6, user_id=5b9c9f9b19974d6fa9abbe87651fac9d, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:32.821463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e033f3b-eae3-4ca4-a864-55d89057f506 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:32.835011 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1344/2712] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:37:32 2022] PUT /identity/v3/projects/2cba9b74bb6b4316a270c1bd7e0a59b6/users/5b9c9f9b19974d6fa9abbe87651fac9d/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:37:32.905259 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c9c8f-8a4c-4814-a45b-ac7dc6d43fe9 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:32.905544 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c9c8f-8a4c-4814-a45b-ac7dc6d43fe9 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:32.905841 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c9c8f-8a4c-4814-a45b-ac7dc6d43fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:32.933762 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-0c5c9c8f-8a4c-4814-a45b-ac7dc6d43fe9 None None] MFA Rules not processed for user `5b9c9f9b19974d6fa9abbe87651fac9d`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:37:32.934971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0c5c9c8f-8a4c-4814-a45b-ac7dc6d43fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:32.998768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1369/2713] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:37:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:37:33.010757 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e033f3b-eae3-4ca4-a864-55d89057f506 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:37:33.012437 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2e033f3b-eae3-4ca4-a864-55d89057f506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:37:33.070629 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e033f3b-eae3-4ca4-a864-55d89057f506 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:37:33.074895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ea6f7-aa98-4b25-bc8e-85d3ef62596b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52470b620abd4cd29ab7e4752f285bb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:37:33.075633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ea6f7-aa98-4b25-bc8e-85d3ef62596b service glance] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:37:33.075914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ea6f7-aa98-4b25-bc8e-85d3ef62596b service glance] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:37:33.076232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ea6f7-aa98-4b25-bc8e-85d3ef62596b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:37:33.081998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:33.086283 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ea6f7-aa98-4b25-bc8e-85d3ef62596b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:37:33.088655 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ea6f7-aa98-4b25-bc8e-85d3ef62596b service glance] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:37:33.099039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1345/2714] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:37:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:37:42.995887 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:37:43.099813 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:03.980517 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c9c8f-8a4c-4814-a45b-ac7dc6d43fe9 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:03.981969 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-0c5c9c8f-8a4c-4814-a45b-ac7dc6d43fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:04.129968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c9c8f-8a4c-4814-a45b-ac7dc6d43fe9 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:04.135884 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d5014-f96b-4aaf-89f9-2eca0bfae5d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:04.136473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d5014-f96b-4aaf-89f9-2eca0bfae5d9 service placement] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:04.136779 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d5014-f96b-4aaf-89f9-2eca0bfae5d9 service placement] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:04.137044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d5014-f96b-4aaf-89f9-2eca0bfae5d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:04.138966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-178d5014-f96b-4aaf-89f9-2eca0bfae5d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:04.151378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d5014-f96b-4aaf-89f9-2eca0bfae5d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:04.153028 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178d5014-f96b-4aaf-89f9-2eca0bfae5d9 service placement] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:04.162700 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1370/2715] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:06.412590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ea6f7-aa98-4b25-bc8e-85d3ef62596b service glance] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:06.414839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-962ea6f7-aa98-4b25-bc8e-85d3ef62596b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:06.534500 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ea6f7-aa98-4b25-bc8e-85d3ef62596b service glance] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:06.539342 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:06.539969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:06.540248 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:06.540596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] PATH_INFO: `/v3/users/48eef76919bd4d25a26e8fa49c1b62e1` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:06.541696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=48eef76919bd4d25a26e8fa49c1b62e1)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:06.543635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:06.593208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48eef76919bd4d25a26e8fa49c1b62e1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:06.593863 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48eef76919bd4d25a26e8fa49c1b62e1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:06.607787 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48eef76919bd4d25a26e8fa49c1b62e1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:06.615551 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48eef76919bd4d25a26e8fa49c1b62e1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:06.623089 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48eef76919bd4d25a26e8fa49c1b62e1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:06.629315 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1346/2716] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:06 2022] DELETE /identity/v3/users/48eef76919bd4d25a26e8fa49c1b62e1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:06.633926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b461aee-56c7-4328-9c2d-3f467e0c5b46 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:06.634406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b461aee-56c7-4328-9c2d-3f467e0c5b46 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:06.634536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b461aee-56c7-4328-9c2d-3f467e0c5b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:06.662433 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6b461aee-56c7-4328-9c2d-3f467e0c5b46 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:06.664207 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6b461aee-56c7-4328-9c2d-3f467e0c5b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:06.727267 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1371/2717] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:06.737597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:06.739490 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:06.794300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:06.846326 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322f6d3-467a-4003-be5e-f0fa8cf99f4e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:06.851918 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913162b-f839-41c9-89bf-860701adb4f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:06.852622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913162b-f839-41c9-89bf-860701adb4f2 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:06.852898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913162b-f839-41c9-89bf-860701adb4f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:06.853165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913162b-f839-41c9-89bf-860701adb4f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:06.863500 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913162b-f839-41c9-89bf-860701adb4f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:06.864974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913162b-f839-41c9-89bf-860701adb4f2 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:06.874670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1347/2718] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:07.162185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b461aee-56c7-4328-9c2d-3f467e0c5b46 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:07.164055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6b461aee-56c7-4328-9c2d-3f467e0c5b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:07.177906 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b461aee-56c7-4328-9c2d-3f467e0c5b46 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:07.183627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:07.184425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:07.184905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:07.185329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] PATH_INFO: `/v3/projects/0f07ca922731461fbcaa0b77cd88596a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:07.186719 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f07ca922731461fbcaa0b77cd88596a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:07.188895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:07.203681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:07.240641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f07ca922731461fbcaa0b77cd88596a', 'request_id': 'req-56903a57-2297-41a4-8a12-32f455fe0fe3'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.246984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f07ca922731461fbcaa0b77cd88596a', 'request_id': 'req-56903a57-2297-41a4-8a12-32f455fe0fe3'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.253451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f07ca922731461fbcaa0b77cd88596a', 'request_id': 'req-56903a57-2297-41a4-8a12-32f455fe0fe3'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.270190 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] The token cache is being invalidate because project 0f07ca922731461fbcaa0b77cd88596a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:07.270496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.272857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.274703 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1372/2719] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:07 2022] DELETE /identity/v3/projects/0f07ca922731461fbcaa0b77cd88596a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:07.278678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913162b-f839-41c9-89bf-860701adb4f2 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:07.289948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913162b-f839-41c9-89bf-860701adb4f2 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:07.353556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:07.354809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:07.355083 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:07.355352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] PATH_INFO: `/v3/users/0bcba283da3c4ecb966f50e3b249f0b3` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:07.356712 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bcba283da3c4ecb966f50e3b249f0b3)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:07.361306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:07.417746 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bcba283da3c4ecb966f50e3b249f0b3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.418443 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bcba283da3c4ecb966f50e3b249f0b3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.432638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bcba283da3c4ecb966f50e3b249f0b3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.440714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bcba283da3c4ecb966f50e3b249f0b3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.450149 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d0d255e9-1102-41d2-96b0-71aaec8a84a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bcba283da3c4ecb966f50e3b249f0b3'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.456526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1348/2720] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:07 2022] DELETE /identity/v3/users/0bcba283da3c4ecb966f50e3b249f0b3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:07.733502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:07.734970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:07.748436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56903a57-2297-41a4-8a12-32f455fe0fe3 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:07.807092 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:07.807732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:07.808008 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:07.808294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] PATH_INFO: `/v3/projects/7969edd74f7f4a778b6549b7012f1b0a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:07.809432 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7969edd74f7f4a778b6549b7012f1b0a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:07.811436 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:07.863704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7969edd74f7f4a778b6549b7012f1b0a', 'request_id': 'req-92aab752-cffc-4a96-8fbc-204e91efcfb0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.870065 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7969edd74f7f4a778b6549b7012f1b0a', 'request_id': 'req-92aab752-cffc-4a96-8fbc-204e91efcfb0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.878112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7969edd74f7f4a778b6549b7012f1b0a', 'request_id': 'req-92aab752-cffc-4a96-8fbc-204e91efcfb0'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.893896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] The token cache is being invalidate because project 7969edd74f7f4a778b6549b7012f1b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:07.894011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.894600 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-92aab752-cffc-4a96-8fbc-204e91efcfb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:07.896202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1373/2721] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:07 2022] DELETE /identity/v3/projects/7969edd74f7f4a778b6549b7012f1b0a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:07.902293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2800f-4f45-4b2f-8cf0-ed9e447f4185 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:07.902793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2800f-4f45-4b2f-8cf0-ed9e447f4185 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:07.903078 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2800f-4f45-4b2f-8cf0-ed9e447f4185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:07.956130 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-44f2800f-4f45-4b2f-8cf0-ed9e447f4185 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:07.957387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-44f2800f-4f45-4b2f-8cf0-ed9e447f4185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:07.962765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:08.019950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1349/2722] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:08.028236 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe35250-84bc-443a-ab14-dfbd96267a6b None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.028598 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe35250-84bc-443a-ab14-dfbd96267a6b None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.028813 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe35250-84bc-443a-ab14-dfbd96267a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.050985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ffe35250-84bc-443a-ab14-dfbd96267a6b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:08.052416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ffe35250-84bc-443a-ab14-dfbd96267a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:08.062381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1374/2723] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:08.066971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2800f-4f45-4b2f-8cf0-ed9e447f4185 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.079948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2800f-4f45-4b2f-8cf0-ed9e447f4185 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.085096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6c6bf-f0a8-4c88-a3a1-147bb1eb4dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.086202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6c6bf-f0a8-4c88-a3a1-147bb1eb4dbf admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.086767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6c6bf-f0a8-4c88-a3a1-147bb1eb4dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.087053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6c6bf-f0a8-4c88-a3a1-147bb1eb4dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.087721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6c6bf-f0a8-4c88-a3a1-147bb1eb4dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:38:08.088029 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6c6bf-f0a8-4c88-a3a1-147bb1eb4dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.089967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6c6bf-f0a8-4c88-a3a1-147bb1eb4dbf admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.104738 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1350/2724] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:38:08.109724 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe35250-84bc-443a-ab14-dfbd96267a6b None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.125414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe35250-84bc-443a-ab14-dfbd96267a6b None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.130794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a38e1d-5fb1-4363-a6a2-c4a6c1f58b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.131424 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a38e1d-5fb1-4363-a6a2-c4a6c1f58b1f admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.132296 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a38e1d-5fb1-4363-a6a2-c4a6c1f58b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.132633 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a38e1d-5fb1-4363-a6a2-c4a6c1f58b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.133452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a38e1d-5fb1-4363-a6a2-c4a6c1f58b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.135529 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a38e1d-5fb1-4363-a6a2-c4a6c1f58b1f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.171464 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:08.183060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1375/2725] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:08 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:08.184393 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6c6bf-f0a8-4c88-a3a1-147bb1eb4dbf admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.199387 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6c6bf-f0a8-4c88-a3a1-147bb1eb4dbf admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.279636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96026ad-854d-48d1-a51d-ee272f76656b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.280239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96026ad-854d-48d1-a51d-ee272f76656b admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.280549 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96026ad-854d-48d1-a51d-ee272f76656b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.280819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96026ad-854d-48d1-a51d-ee272f76656b admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.281436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96026ad-854d-48d1-a51d-ee272f76656b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.283375 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96026ad-854d-48d1-a51d-ee272f76656b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.305745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1351/2726] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:08 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:08.309714 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a38e1d-5fb1-4363-a6a2-c4a6c1f58b1f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.319850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a38e1d-5fb1-4363-a6a2-c4a6c1f58b1f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.324828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd3e8e-87b3-4551-ac4d-246d4a55b909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.325422 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd3e8e-87b3-4551-ac4d-246d4a55b909 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.325702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd3e8e-87b3-4551-ac4d-246d4a55b909 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.325974 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd3e8e-87b3-4551-ac4d-246d4a55b909 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.326636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd3e8e-87b3-4551-ac4d-246d4a55b909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.328255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fd3e8e-87b3-4551-ac4d-246d4a55b909 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.338414 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1376/2727] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:08.342524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96026ad-854d-48d1-a51d-ee272f76656b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.352892 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96026ad-854d-48d1-a51d-ee272f76656b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.357681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8987c42-eaba-496e-b21a-3e28e41f192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.358252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8987c42-eaba-496e-b21a-3e28e41f192e admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.358525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8987c42-eaba-496e-b21a-3e28e41f192e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.358792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8987c42-eaba-496e-b21a-3e28e41f192e admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.359425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8987c42-eaba-496e-b21a-3e28e41f192e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.361098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8987c42-eaba-496e-b21a-3e28e41f192e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.372484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1352/2728] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:08.376650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd3e8e-87b3-4551-ac4d-246d4a55b909 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.387275 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd3e8e-87b3-4551-ac4d-246d4a55b909 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.392037 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a6b01-10ef-4116-8341-8756e3fdc386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.392717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a6b01-10ef-4116-8341-8756e3fdc386 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.392999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a6b01-10ef-4116-8341-8756e3fdc386 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.393273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a6b01-10ef-4116-8341-8756e3fdc386 admin admin] PATH_INFO: `/v3/projects/baf19f3b9d0b463b8ac746ec16ab96aa/users/687d64124a3c4d36bba3099701ec62fc/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.408042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a6b01-10ef-4116-8341-8756e3fdc386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf19f3b9d0b463b8ac746ec16ab96aa, user_id=687d64124a3c4d36bba3099701ec62fc, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.410744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a6b01-10ef-4116-8341-8756e3fdc386 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.424663 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1377/2729] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:08 2022] PUT /identity/v3/projects/baf19f3b9d0b463b8ac746ec16ab96aa/users/687d64124a3c4d36bba3099701ec62fc/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:08.435611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196de9d-b9c1-46cc-984b-73f95ff11faa None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.436107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196de9d-b9c1-46cc-984b-73f95ff11faa None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.436226 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196de9d-b9c1-46cc-984b-73f95ff11faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.464804 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-4196de9d-b9c1-46cc-984b-73f95ff11faa None None] MFA Rules not processed for user `687d64124a3c4d36bba3099701ec62fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:08.466030 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4196de9d-b9c1-46cc-984b-73f95ff11faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:08.537166 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1353/2730] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:38:08.542838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a6b01-10ef-4116-8341-8756e3fdc386 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.609677 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a6b01-10ef-4116-8341-8756e3fdc386 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.615157 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f18826-e30c-47e8-abfa-ab50a9068861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.615788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f18826-e30c-47e8-abfa-ab50a9068861 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.616110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f18826-e30c-47e8-abfa-ab50a9068861 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.616374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f18826-e30c-47e8-abfa-ab50a9068861 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.617049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f18826-e30c-47e8-abfa-ab50a9068861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.619044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f18826-e30c-47e8-abfa-ab50a9068861 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.648901 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1378/2731] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:08 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:08.653088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196de9d-b9c1-46cc-984b-73f95ff11faa None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.665278 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196de9d-b9c1-46cc-984b-73f95ff11faa None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.669049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:08.726985 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa28505-811a-4f1b-a91c-4ecc8ad18d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.727608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa28505-811a-4f1b-a91c-4ecc8ad18d05 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.727891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa28505-811a-4f1b-a91c-4ecc8ad18d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.728164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa28505-811a-4f1b-a91c-4ecc8ad18d05 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.728833 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa28505-811a-4f1b-a91c-4ecc8ad18d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.730767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa28505-811a-4f1b-a91c-4ecc8ad18d05 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.755987 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1354/2732] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:08 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:08.758349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f18826-e30c-47e8-abfa-ab50a9068861 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.768700 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f18826-e30c-47e8-abfa-ab50a9068861 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.774878 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38015f3-31d9-4f78-95ac-30b2b9e78821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.775481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38015f3-31d9-4f78-95ac-30b2b9e78821 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.775764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38015f3-31d9-4f78-95ac-30b2b9e78821 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.776036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38015f3-31d9-4f78-95ac-30b2b9e78821 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.776733 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38015f3-31d9-4f78-95ac-30b2b9e78821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.778347 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38015f3-31d9-4f78-95ac-30b2b9e78821 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.793888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1379/2733] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:38:08.795032 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa28505-811a-4f1b-a91c-4ecc8ad18d05 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.805681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa28505-811a-4f1b-a91c-4ecc8ad18d05 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.810508 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7c07c-7985-4702-9962-0144028916fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.811190 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7c07c-7985-4702-9962-0144028916fc admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.811470 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7c07c-7985-4702-9962-0144028916fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.811745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7c07c-7985-4702-9962-0144028916fc admin admin] PATH_INFO: `/v3/projects/23fb947fbc8040bd9a13ec55e5ed0d3a/users/faf7e0bab1f942398d2e106dcdccfb1b/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.826980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7c07c-7985-4702-9962-0144028916fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=23fb947fbc8040bd9a13ec55e5ed0d3a, user_id=faf7e0bab1f942398d2e106dcdccfb1b, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.829724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7c07c-7985-4702-9962-0144028916fc admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.843126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1355/2734] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:08 2022] PUT /identity/v3/projects/23fb947fbc8040bd9a13ec55e5ed0d3a/users/faf7e0bab1f942398d2e106dcdccfb1b/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:08.847026 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38015f3-31d9-4f78-95ac-30b2b9e78821 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.857708 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38015f3-31d9-4f78-95ac-30b2b9e78821 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.915231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cd478-b1e0-4537-8ae2-bbbf7c8f7093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.915744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cd478-b1e0-4537-8ae2-bbbf7c8f7093 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.916040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cd478-b1e0-4537-8ae2-bbbf7c8f7093 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.916307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cd478-b1e0-4537-8ae2-bbbf7c8f7093 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.917075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cd478-b1e0-4537-8ae2-bbbf7c8f7093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.918693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cd478-b1e0-4537-8ae2-bbbf7c8f7093 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.928426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1380/2735] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:08.932615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7c07c-7985-4702-9962-0144028916fc admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:08.942745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7c07c-7985-4702-9962-0144028916fc admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:08.948786 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6600dc-ff70-4f57-add0-a5bbf6df8107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:08.949428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6600dc-ff70-4f57-add0-a5bbf6df8107 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.949704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6600dc-ff70-4f57-add0-a5bbf6df8107 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.949975 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6600dc-ff70-4f57-add0-a5bbf6df8107 admin admin] PATH_INFO: `/v3/projects/23fb947fbc8040bd9a13ec55e5ed0d3a/users/faf7e0bab1f942398d2e106dcdccfb1b/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:08.951878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6600dc-ff70-4f57-add0-a5bbf6df8107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23fb947fbc8040bd9a13ec55e5ed0d3a, user_id=faf7e0bab1f942398d2e106dcdccfb1b, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:08.954559 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6600dc-ff70-4f57-add0-a5bbf6df8107 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:08.965367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1356/2736] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:08 2022] PUT /identity/v3/projects/23fb947fbc8040bd9a13ec55e5ed0d3a/users/faf7e0bab1f942398d2e106dcdccfb1b/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:08.976365 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db88b8-6957-4995-8389-d4bc172e5746 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:08.976688 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db88b8-6957-4995-8389-d4bc172e5746 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:08.976968 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db88b8-6957-4995-8389-d4bc172e5746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:09.003850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e4db88b8-6957-4995-8389-d4bc172e5746 None None] MFA Rules not processed for user `faf7e0bab1f942398d2e106dcdccfb1b`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:09.005112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e4db88b8-6957-4995-8389-d4bc172e5746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:09.029412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:09.074349 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1381/2737] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:09.086377 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6600dc-ff70-4f57-add0-a5bbf6df8107 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:09.088066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-cd6600dc-ff70-4f57-add0-a5bbf6df8107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:09.145611 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6600dc-ff70-4f57-add0-a5bbf6df8107 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:09.151848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3d033-1eba-466f-8507-5cb94d1a380c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:09.152455 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3d033-1eba-466f-8507-5cb94d1a380c service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:09.152761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3d033-1eba-466f-8507-5cb94d1a380c service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:09.153035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3d033-1eba-466f-8507-5cb94d1a380c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:09.162107 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3d033-1eba-466f-8507-5cb94d1a380c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:09.163649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3d033-1eba-466f-8507-5cb94d1a380c service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:09.173082 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1357/2738] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:09.217960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db88b8-6957-4995-8389-d4bc172e5746 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:09.219353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e4db88b8-6957-4995-8389-d4bc172e5746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:09.232384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db88b8-6957-4995-8389-d4bc172e5746 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:09.238003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8eec8-0e25-4397-8280-38fc91cf10c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:09.238739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8eec8-0e25-4397-8280-38fc91cf10c0 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:09.239023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8eec8-0e25-4397-8280-38fc91cf10c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:09.239293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8eec8-0e25-4397-8280-38fc91cf10c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:09.298200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8eec8-0e25-4397-8280-38fc91cf10c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:09.299635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8eec8-0e25-4397-8280-38fc91cf10c0 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:09.308625 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1382/2739] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:38:12.609521 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3d033-1eba-466f-8507-5cb94d1a380c service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:12.672978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3d033-1eba-466f-8507-5cb94d1a380c service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:12.677542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:12.678274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:12.678560 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:12.678835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] PATH_INFO: `/v3/users/687d64124a3c4d36bba3099701ec62fc` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:12.679965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=687d64124a3c4d36bba3099701ec62fc)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:12.681926 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:12.684234 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:12.730575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687d64124a3c4d36bba3099701ec62fc'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:12.731236 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687d64124a3c4d36bba3099701ec62fc'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:12.743955 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687d64124a3c4d36bba3099701ec62fc'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:12.750176 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687d64124a3c4d36bba3099701ec62fc'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:12.757869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687d64124a3c4d36bba3099701ec62fc'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:12.763940 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1358/2740] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:12 2022] DELETE /identity/v3/users/687d64124a3c4d36bba3099701ec62fc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:12.768612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d1774-01a0-4a10-99b2-00c5ea8fa6a2 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:12.769070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d1774-01a0-4a10-99b2-00c5ea8fa6a2 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:12.769243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d1774-01a0-4a10-99b2-00c5ea8fa6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:12.790679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-ea0d1774-01a0-4a10-99b2-00c5ea8fa6a2 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:12.792057 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ea0d1774-01a0-4a10-99b2-00c5ea8fa6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:12.859181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1383/2741] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:12.868577 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:12.869950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:12.927553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac26743-3bf8-4d91-b742-e16bc2209408 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:12.931177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463f708-9781-4b5a-92ba-19d2f0069aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:12.931781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463f708-9781-4b5a-92ba-19d2f0069aaa service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:12.932052 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463f708-9781-4b5a-92ba-19d2f0069aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:12.932318 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463f708-9781-4b5a-92ba-19d2f0069aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:12.941540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463f708-9781-4b5a-92ba-19d2f0069aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:12.942983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463f708-9781-4b5a-92ba-19d2f0069aaa service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:12.953901 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1359/2742] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:13.230603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d1774-01a0-4a10-99b2-00c5ea8fa6a2 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:13.232140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-ea0d1774-01a0-4a10-99b2-00c5ea8fa6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:13.259982 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d1774-01a0-4a10-99b2-00c5ea8fa6a2 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:13.266367 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:13.267031 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:13.267305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:13.267572 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] PATH_INFO: `/v3/projects/baf19f3b9d0b463b8ac746ec16ab96aa` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:13.268744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf19f3b9d0b463b8ac746ec16ab96aa)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:13.270701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:13.324358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf19f3b9d0b463b8ac746ec16ab96aa', 'request_id': 'req-bedc8a6d-2a68-490f-92bd-faab37a3b157'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.329885 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:13.332277 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf19f3b9d0b463b8ac746ec16ab96aa', 'request_id': 'req-bedc8a6d-2a68-490f-92bd-faab37a3b157'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.339757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf19f3b9d0b463b8ac746ec16ab96aa', 'request_id': 'req-bedc8a6d-2a68-490f-92bd-faab37a3b157'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.363758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] The token cache is being invalidate because project baf19f3b9d0b463b8ac746ec16ab96aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:13.364070 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.364733 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.367568 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1384/2743] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:13 2022] DELETE /identity/v3/projects/baf19f3b9d0b463b8ac746ec16ab96aa => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:13.371534 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463f708-9781-4b5a-92ba-19d2f0069aaa service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:13.382400 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463f708-9781-4b5a-92ba-19d2f0069aaa service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:13.469121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:13.469754 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:13.470037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:13.470322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] PATH_INFO: `/v3/users/faf7e0bab1f942398d2e106dcdccfb1b` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:13.471585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf7e0bab1f942398d2e106dcdccfb1b)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:13.473760 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:13.506466 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:13.528297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf7e0bab1f942398d2e106dcdccfb1b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.528997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf7e0bab1f942398d2e106dcdccfb1b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.541685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf7e0bab1f942398d2e106dcdccfb1b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.548598 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf7e0bab1f942398d2e106dcdccfb1b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.557360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-e44b07db-696b-4d73-8318-7708cb913d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf7e0bab1f942398d2e106dcdccfb1b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:13.564014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1360/2744] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:13 2022] DELETE /identity/v3/users/faf7e0bab1f942398d2e106dcdccfb1b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:38:13.856659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:13.858184 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:13.872619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc8a6d-2a68-490f-92bd-faab37a3b157 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:13.942698 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:13.943429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:13.943743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:13.944915 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] PATH_INFO: `/v3/projects/23fb947fbc8040bd9a13ec55e5ed0d3a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:13.946199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] RBAC: Authorizing `identity:delete_project(project_id=23fb947fbc8040bd9a13ec55e5ed0d3a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:13.948444 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:13.999080 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23fb947fbc8040bd9a13ec55e5ed0d3a', 'request_id': 'req-26b51964-fd21-47c8-b206-b7a0707f367b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:14.004743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23fb947fbc8040bd9a13ec55e5ed0d3a', 'request_id': 'req-26b51964-fd21-47c8-b206-b7a0707f367b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:14.013053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23fb947fbc8040bd9a13ec55e5ed0d3a', 'request_id': 'req-26b51964-fd21-47c8-b206-b7a0707f367b'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:14.026472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] The token cache is being invalidate because project 23fb947fbc8040bd9a13ec55e5ed0d3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:14.026768 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:14.027396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-26b51964-fd21-47c8-b206-b7a0707f367b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:14.029035 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1385/2745] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:13 2022] DELETE /identity/v3/projects/23fb947fbc8040bd9a13ec55e5ed0d3a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:14.033954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70a6b0-8fca-4385-be90-ff1eb9680a06 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.034473 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70a6b0-8fca-4385-be90-ff1eb9680a06 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.034582 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70a6b0-8fca-4385-be90-ff1eb9680a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.054981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ef70a6b0-8fca-4385-be90-ff1eb9680a06 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:14.056906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ef70a6b0-8fca-4385-be90-ff1eb9680a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:14.116434 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1361/2746] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:14.124182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e4250-d9eb-4e11-a9f1-8ef51ed49a5f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.124510 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e4250-d9eb-4e11-a9f1-8ef51ed49a5f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.124796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e4250-d9eb-4e11-a9f1-8ef51ed49a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.144346 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-e37e4250-d9eb-4e11-a9f1-8ef51ed49a5f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:14.145570 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e37e4250-d9eb-4e11-a9f1-8ef51ed49a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:14.157986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1386/2747] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:14.162317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70a6b0-8fca-4385-be90-ff1eb9680a06 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.172659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70a6b0-8fca-4385-be90-ff1eb9680a06 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.177186 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98840e-85c5-4eb9-9862-04901552b3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.177777 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98840e-85c5-4eb9-9862-04901552b3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.178068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98840e-85c5-4eb9-9862-04901552b3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.178317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98840e-85c5-4eb9-9862-04901552b3b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.178951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98840e-85c5-4eb9-9862-04901552b3b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:38:14.179247 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98840e-85c5-4eb9-9862-04901552b3b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.180868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98840e-85c5-4eb9-9862-04901552b3b3 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.192202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1362/2748] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:38:14.196984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e4250-d9eb-4e11-a9f1-8ef51ed49a5f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.206662 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e4250-d9eb-4e11-a9f1-8ef51ed49a5f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.212310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b730384-29ee-4cfe-b0e8-23573dbc034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.212923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b730384-29ee-4cfe-b0e8-23573dbc034d admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.213197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b730384-29ee-4cfe-b0e8-23573dbc034d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.213461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b730384-29ee-4cfe-b0e8-23573dbc034d admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.214079 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b730384-29ee-4cfe-b0e8-23573dbc034d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.215955 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b730384-29ee-4cfe-b0e8-23573dbc034d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.244177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1387/2749] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:14 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:14.248413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98840e-85c5-4eb9-9862-04901552b3b3 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.258957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98840e-85c5-4eb9-9862-04901552b3b3 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.313869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1685b-0728-4f68-80a4-b18c25e6425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.314472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1685b-0728-4f68-80a4-b18c25e6425e admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.314750 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1685b-0728-4f68-80a4-b18c25e6425e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.315021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1685b-0728-4f68-80a4-b18c25e6425e admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.315636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1685b-0728-4f68-80a4-b18c25e6425e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.317597 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1685b-0728-4f68-80a4-b18c25e6425e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.340418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1363/2750] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:14 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:14.344472 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b730384-29ee-4cfe-b0e8-23573dbc034d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.354978 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b730384-29ee-4cfe-b0e8-23573dbc034d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.359645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93977955-030f-4246-8c53-1035fff0c3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.360237 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93977955-030f-4246-8c53-1035fff0c3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.360548 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93977955-030f-4246-8c53-1035fff0c3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.360821 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93977955-030f-4246-8c53-1035fff0c3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.361455 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93977955-030f-4246-8c53-1035fff0c3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.363059 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93977955-030f-4246-8c53-1035fff0c3c8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.373329 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1388/2751] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:14.377257 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1685b-0728-4f68-80a4-b18c25e6425e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.387851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1685b-0728-4f68-80a4-b18c25e6425e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.392853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d96889-e42e-4a10-864d-8d33778994a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.393553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d96889-e42e-4a10-864d-8d33778994a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.393838 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d96889-e42e-4a10-864d-8d33778994a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.394142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d96889-e42e-4a10-864d-8d33778994a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.394878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d96889-e42e-4a10-864d-8d33778994a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.396627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d96889-e42e-4a10-864d-8d33778994a6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.408636 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1364/2752] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:14.413471 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93977955-030f-4246-8c53-1035fff0c3c8 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.423332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93977955-030f-4246-8c53-1035fff0c3c8 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.427877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9b17f-4fb0-49bd-806c-b65f9ded3aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.428531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b17f-4fb0-49bd-806c-b65f9ded3aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.428814 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b17f-4fb0-49bd-806c-b65f9ded3aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.429090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b17f-4fb0-49bd-806c-b65f9ded3aaa admin admin] PATH_INFO: `/v3/projects/0814baf2ffe348b98cd011544df83c0e/users/9c650f95287048eda94fb368520c5686/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.444378 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9b17f-4fb0-49bd-806c-b65f9ded3aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0814baf2ffe348b98cd011544df83c0e, user_id=9c650f95287048eda94fb368520c5686, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.447124 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9b17f-4fb0-49bd-806c-b65f9ded3aaa admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.460684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1389/2753] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:14 2022] PUT /identity/v3/projects/0814baf2ffe348b98cd011544df83c0e/users/9c650f95287048eda94fb368520c5686/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:38:14.470816 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbc577-c33a-4d48-8155-71117b52d03c None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.471102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbc577-c33a-4d48-8155-71117b52d03c None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.471380 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbc577-c33a-4d48-8155-71117b52d03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.491113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:14.503323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-7cfbc577-c33a-4d48-8155-71117b52d03c None None] MFA Rules not processed for user `9c650f95287048eda94fb368520c5686`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:14.505094 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7cfbc577-c33a-4d48-8155-71117b52d03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:14.577974 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1365/2754] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:14.582439 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9b17f-4fb0-49bd-806c-b65f9ded3aaa admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.592776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:14.650577 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9b17f-4fb0-49bd-806c-b65f9ded3aaa admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.656321 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe18b6-9d95-405b-9f60-35eb03458b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.656984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe18b6-9d95-405b-9f60-35eb03458b0f admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.657260 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe18b6-9d95-405b-9f60-35eb03458b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.657553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe18b6-9d95-405b-9f60-35eb03458b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.658385 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe18b6-9d95-405b-9f60-35eb03458b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.660820 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe18b6-9d95-405b-9f60-35eb03458b0f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.698838 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1390/2755] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:14 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:14.702721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbc577-c33a-4d48-8155-71117b52d03c None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.714297 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbc577-c33a-4d48-8155-71117b52d03c None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.771965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608a7f9-6123-4c6c-8416-e49acec4cc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.772606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608a7f9-6123-4c6c-8416-e49acec4cc05 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.772883 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608a7f9-6123-4c6c-8416-e49acec4cc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.773154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608a7f9-6123-4c6c-8416-e49acec4cc05 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.773766 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608a7f9-6123-4c6c-8416-e49acec4cc05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.775969 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608a7f9-6123-4c6c-8416-e49acec4cc05 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.798319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1366/2756] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:14 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:38:14.801996 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe18b6-9d95-405b-9f60-35eb03458b0f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.813331 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe18b6-9d95-405b-9f60-35eb03458b0f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.818083 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f39f5-74d8-48b0-8c9d-8f19dfff8bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.819299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f39f5-74d8-48b0-8c9d-8f19dfff8bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.819578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f39f5-74d8-48b0-8c9d-8f19dfff8bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.819862 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f39f5-74d8-48b0-8c9d-8f19dfff8bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.820479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f39f5-74d8-48b0-8c9d-8f19dfff8bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.822185 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f39f5-74d8-48b0-8c9d-8f19dfff8bc0 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.834089 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1391/2757] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:14.838426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608a7f9-6123-4c6c-8416-e49acec4cc05 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.848444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608a7f9-6123-4c6c-8416-e49acec4cc05 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.853907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6a2bb-9365-401b-b102-9b32f58672cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.854667 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6a2bb-9365-401b-b102-9b32f58672cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.854978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6a2bb-9365-401b-b102-9b32f58672cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.855327 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6a2bb-9365-401b-b102-9b32f58672cc admin admin] PATH_INFO: `/v3/projects/81fc1692609642c6ae36457f5da656d2/users/2a330260610b4f0396e13a8dd0660b78/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.872353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6a2bb-9365-401b-b102-9b32f58672cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=81fc1692609642c6ae36457f5da656d2, user_id=2a330260610b4f0396e13a8dd0660b78, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.875142 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6a2bb-9365-401b-b102-9b32f58672cc admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.889364 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1367/2758] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:14 2022] PUT /identity/v3/projects/81fc1692609642c6ae36457f5da656d2/users/2a330260610b4f0396e13a8dd0660b78/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:14.893239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f39f5-74d8-48b0-8c9d-8f19dfff8bc0 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.903380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f39f5-74d8-48b0-8c9d-8f19dfff8bc0 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.960787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54117d81-e713-419d-9961-75d432ee10fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.961307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54117d81-e713-419d-9961-75d432ee10fc admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.961581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54117d81-e713-419d-9961-75d432ee10fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.961856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54117d81-e713-419d-9961-75d432ee10fc admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.962487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54117d81-e713-419d-9961-75d432ee10fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.964111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54117d81-e713-419d-9961-75d432ee10fc admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:14.973883 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1392/2759] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:14.978654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6a2bb-9365-401b-b102-9b32f58672cc admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:14.988697 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6a2bb-9365-401b-b102-9b32f58672cc admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:14.994024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba7439-4725-469f-a2bd-103b1f18944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:14.994659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba7439-4725-469f-a2bd-103b1f18944d admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:14.994931 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba7439-4725-469f-a2bd-103b1f18944d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:14.995199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba7439-4725-469f-a2bd-103b1f18944d admin admin] PATH_INFO: `/v3/projects/81fc1692609642c6ae36457f5da656d2/users/2a330260610b4f0396e13a8dd0660b78/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:14.997091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba7439-4725-469f-a2bd-103b1f18944d admin admin] RBAC: Authorizing `identity:create_grant(project_id=81fc1692609642c6ae36457f5da656d2, user_id=2a330260610b4f0396e13a8dd0660b78, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:14.999707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba7439-4725-469f-a2bd-103b1f18944d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:15.009897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1368/2760] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:14 2022] PUT /identity/v3/projects/81fc1692609642c6ae36457f5da656d2/users/2a330260610b4f0396e13a8dd0660b78/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:15.020722 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c3134-440a-4b57-9022-013af4e517ff None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:15.021006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c3134-440a-4b57-9022-013af4e517ff None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:15.021280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c3134-440a-4b57-9022-013af4e517ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:15.048827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-635c3134-440a-4b57-9022-013af4e517ff None None] MFA Rules not processed for user `2a330260610b4f0396e13a8dd0660b78`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:15.050024 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-635c3134-440a-4b57-9022-013af4e517ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:15.117430 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1393/2761] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:15.127898 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba7439-4725-469f-a2bd-103b1f18944d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:15.129628 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-72ba7439-4725-469f-a2bd-103b1f18944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:15.184414 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:15.208350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba7439-4725-469f-a2bd-103b1f18944d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:15.212102 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f071af-1050-4e0a-89ab-db882805d925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:15.212674 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f071af-1050-4e0a-89ab-db882805d925 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:15.212950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f071af-1050-4e0a-89ab-db882805d925 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:15.213219 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f071af-1050-4e0a-89ab-db882805d925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:15.268905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f071af-1050-4e0a-89ab-db882805d925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:15.270355 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f071af-1050-4e0a-89ab-db882805d925 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:15.279460 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1369/2762] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:19.168995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c3134-440a-4b57-9022-013af4e517ff None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:19.170419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-635c3134-440a-4b57-9022-013af4e517ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:19.176725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:19.248679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c3134-440a-4b57-9022-013af4e517ff None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:19.250805 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:19.251392 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:19.251668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:19.251933 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] PATH_INFO: `/v3/users/5b9c9f9b19974d6fa9abbe87651fac9d` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:19.253097 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b9c9f9b19974d6fa9abbe87651fac9d)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:19.255169 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:19.300736 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b9c9f9b19974d6fa9abbe87651fac9d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:19.301391 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b9c9f9b19974d6fa9abbe87651fac9d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:19.310268 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b9c9f9b19974d6fa9abbe87651fac9d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:19.317764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b9c9f9b19974d6fa9abbe87651fac9d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:19.319526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b9c9f9b19974d6fa9abbe87651fac9d'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:19.338526 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1394/2763] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:19 2022] DELETE /identity/v3/users/5b9c9f9b19974d6fa9abbe87651fac9d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:19.343575 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096e5b-4e90-4bf3-98d1-0c6f3b87a8b4 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:19.344066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096e5b-4e90-4bf3-98d1-0c6f3b87a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:19.344202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096e5b-4e90-4bf3-98d1-0c6f3b87a8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:19.364939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-f7096e5b-4e90-4bf3-98d1-0c6f3b87a8b4 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:19.366728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f7096e5b-4e90-4bf3-98d1-0c6f3b87a8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:19.426382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1370/2764] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:19.436806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:19.438218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:19.497416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752606b4-45ad-4a62-8811-2926fad9f052 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:19.501199 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc78ca5-3b68-4194-a6fe-ecd69e61174e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:19.501743 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc78ca5-3b68-4194-a6fe-ecd69e61174e service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:19.502020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc78ca5-3b68-4194-a6fe-ecd69e61174e service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:19.502291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc78ca5-3b68-4194-a6fe-ecd69e61174e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:19.511138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc78ca5-3b68-4194-a6fe-ecd69e61174e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:19.512782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc78ca5-3b68-4194-a6fe-ecd69e61174e service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:19.522361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1395/2765] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:20.058580 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096e5b-4e90-4bf3-98d1-0c6f3b87a8b4 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:20.060075 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f7096e5b-4e90-4bf3-98d1-0c6f3b87a8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:20.072625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096e5b-4e90-4bf3-98d1-0c6f3b87a8b4 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:20.077172 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:20.077805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.078080 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.078353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] PATH_INFO: `/v3/projects/2cba9b74bb6b4316a270c1bd7e0a59b6` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.079497 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cba9b74bb6b4316a270c1bd7e0a59b6)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:20.081556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:20.131765 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cba9b74bb6b4316a270c1bd7e0a59b6', 'request_id': 'req-02954608-239a-4a57-a134-8c701ff93c63'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:20.138117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cba9b74bb6b4316a270c1bd7e0a59b6', 'request_id': 'req-02954608-239a-4a57-a134-8c701ff93c63'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:20.145488 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cba9b74bb6b4316a270c1bd7e0a59b6', 'request_id': 'req-02954608-239a-4a57-a134-8c701ff93c63'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:20.161349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] The token cache is being invalidate because project 2cba9b74bb6b4316a270c1bd7e0a59b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:20.161625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:20.162869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-02954608-239a-4a57-a134-8c701ff93c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:20.164415 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1371/2766] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:20 2022] DELETE /identity/v3/projects/2cba9b74bb6b4316a270c1bd7e0a59b6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:20.169794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4e600-cb3b-4579-a716-56c2ec091996 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.170351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4e600-cb3b-4579-a716-56c2ec091996 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.170351 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4e600-cb3b-4579-a716-56c2ec091996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.191510 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6be4e600-cb3b-4579-a716-56c2ec091996 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:20.192877 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6be4e600-cb3b-4579-a716-56c2ec091996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:20.219110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:20.273828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1396/2767] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:20.281463 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69030-0271-431b-b4fd-bb886de67e43 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.281751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69030-0271-431b-b4fd-bb886de67e43 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.282025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69030-0271-431b-b4fd-bb886de67e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.302768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-22f69030-0271-431b-b4fd-bb886de67e43 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:20.304777 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-22f69030-0271-431b-b4fd-bb886de67e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:20.313797 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1372/2768] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:20.318264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4e600-cb3b-4579-a716-56c2ec091996 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:20.328218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4e600-cb3b-4579-a716-56c2ec091996 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:20.332807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264cba0-7cb1-4b12-a155-e58012b76ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:20.333405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264cba0-7cb1-4b12-a155-e58012b76ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.333685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264cba0-7cb1-4b12-a155-e58012b76ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.334002 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264cba0-7cb1-4b12-a155-e58012b76ef5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.334856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264cba0-7cb1-4b12-a155-e58012b76ef5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:38:20.335170 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264cba0-7cb1-4b12-a155-e58012b76ef5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:20.336848 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264cba0-7cb1-4b12-a155-e58012b76ef5 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:20.353039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1397/2769] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:38:20.354285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f69030-0271-431b-b4fd-bb886de67e43 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:20.358742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:20.364670 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f69030-0271-431b-b4fd-bb886de67e43 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:20.372849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bc03d-069b-4d1d-b99f-e2197e1f6222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:20.373840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bc03d-069b-4d1d-b99f-e2197e1f6222 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.374300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bc03d-069b-4d1d-b99f-e2197e1f6222 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.374767 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bc03d-069b-4d1d-b99f-e2197e1f6222 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.375396 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bc03d-069b-4d1d-b99f-e2197e1f6222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:20.377491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bc03d-069b-4d1d-b99f-e2197e1f6222 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:20.408742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1373/2770] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:20 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:20.413334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264cba0-7cb1-4b12-a155-e58012b76ef5 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:20.423861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264cba0-7cb1-4b12-a155-e58012b76ef5 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:20.483944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edec161-33b0-4e9a-9b34-db2b3dc6e540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:20.484579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edec161-33b0-4e9a-9b34-db2b3dc6e540 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.484857 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edec161-33b0-4e9a-9b34-db2b3dc6e540 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.485132 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edec161-33b0-4e9a-9b34-db2b3dc6e540 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.485764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edec161-33b0-4e9a-9b34-db2b3dc6e540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:20.487671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edec161-33b0-4e9a-9b34-db2b3dc6e540 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:20.512543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1398/2771] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:20 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:38:20.516692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bc03d-069b-4d1d-b99f-e2197e1f6222 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:20.527372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bc03d-069b-4d1d-b99f-e2197e1f6222 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:20.532158 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf65c2-f1be-4dac-8f58-4f0baf0ef89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:20.532824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf65c2-f1be-4dac-8f58-4f0baf0ef89c admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.533101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf65c2-f1be-4dac-8f58-4f0baf0ef89c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.533367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf65c2-f1be-4dac-8f58-4f0baf0ef89c admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.534004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf65c2-f1be-4dac-8f58-4f0baf0ef89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:20.535600 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf65c2-f1be-4dac-8f58-4f0baf0ef89c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:20.546290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1374/2772] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:20.550332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edec161-33b0-4e9a-9b34-db2b3dc6e540 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:20.560935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edec161-33b0-4e9a-9b34-db2b3dc6e540 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:20.566778 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c54e5-4ebb-4796-b629-12f00b60354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:20.567463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c54e5-4ebb-4796-b629-12f00b60354a admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.567748 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c54e5-4ebb-4796-b629-12f00b60354a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.568060 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c54e5-4ebb-4796-b629-12f00b60354a admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.569434 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c54e5-4ebb-4796-b629-12f00b60354a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:20.570772 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c54e5-4ebb-4796-b629-12f00b60354a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:20.582440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1399/2773] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:20.587338 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf65c2-f1be-4dac-8f58-4f0baf0ef89c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:20.605899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf65c2-f1be-4dac-8f58-4f0baf0ef89c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:20.615444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42851b1d-8a03-439b-866a-3372c4d05c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:20.616125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42851b1d-8a03-439b-866a-3372c4d05c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.620694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42851b1d-8a03-439b-866a-3372c4d05c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.621061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42851b1d-8a03-439b-866a-3372c4d05c5e admin admin] PATH_INFO: `/v3/projects/0bbc14c088064941bb7d6104305aa82b/users/a2c8189bb1254baf93c2353cea6f5b8d/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.637181 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42851b1d-8a03-439b-866a-3372c4d05c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bbc14c088064941bb7d6104305aa82b, user_id=a2c8189bb1254baf93c2353cea6f5b8d, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:20.639895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42851b1d-8a03-439b-866a-3372c4d05c5e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:20.657858 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1375/2774] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:20 2022] PUT /identity/v3/projects/0bbc14c088064941bb7d6104305aa82b/users/a2c8189bb1254baf93c2353cea6f5b8d/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:38:20.670150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccab7e8-489b-447b-b1ea-cb39ede32e0e None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.670488 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccab7e8-489b-447b-b1ea-cb39ede32e0e None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.670763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccab7e8-489b-447b-b1ea-cb39ede32e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.699639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-6ccab7e8-489b-447b-b1ea-cb39ede32e0e None None] MFA Rules not processed for user `a2c8189bb1254baf93c2353cea6f5b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:20.701004 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-6ccab7e8-489b-447b-b1ea-cb39ede32e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:20.765405 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1400/2775] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:20.777558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42851b1d-8a03-439b-866a-3372c4d05c5e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:20.779222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-42851b1d-8a03-439b-866a-3372c4d05c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:20.836121 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42851b1d-8a03-439b-866a-3372c4d05c5e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:20.839647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ca642-71e0-445a-8862-8837cb66c42a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:20.840244 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca642-71e0-445a-8862-8837cb66c42a service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:20.840544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca642-71e0-445a-8862-8837cb66c42a service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:20.840819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ca642-71e0-445a-8862-8837cb66c42a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:20.848870 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ca642-71e0-445a-8862-8837cb66c42a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:20.851049 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ca642-71e0-445a-8862-8837cb66c42a service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:20.860043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1376/2776] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:38:22.835210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccab7e8-489b-447b-b1ea-cb39ede32e0e None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:22.845011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccab7e8-489b-447b-b1ea-cb39ede32e0e None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:22.848555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:22.849147 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:22.849426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:22.849693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:22.851474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:22.871514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:22.917016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:22.918480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:22.929029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1401/2777] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:23.327324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c84f84-fc37-45fc-9f9d-82e23c11c7c3 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:23.327796 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c84f84-fc37-45fc-9f9d-82e23c11c7c3 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:23.327905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c84f84-fc37-45fc-9f9d-82e23c11c7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:23.351173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-87c84f84-fc37-45fc-9f9d-82e23c11c7c3 None None] MFA Rules not processed for user `7d042aa1a703464eab4a440db9ce1374`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:23.352381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-87c84f84-fc37-45fc-9f9d-82e23c11c7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:23.412413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1377/2778] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:23.654785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:23.656191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:23.715757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b54134-d162-4d71-8c3a-a3f891d98241 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:23.719334 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a5b54d-3556-48d9-9d37-fbfdd85c76a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:23.719930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5b54d-3556-48d9-9d37-fbfdd85c76a5 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:23.720208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5b54d-3556-48d9-9d37-fbfdd85c76a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:23.720486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5b54d-3556-48d9-9d37-fbfdd85c76a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:23.722519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-03a5b54d-3556-48d9-9d37-fbfdd85c76a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:23.732291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5b54d-3556-48d9-9d37-fbfdd85c76a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:23.733766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a5b54d-3556-48d9-9d37-fbfdd85c76a5 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:23.743421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1402/2779] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:24.478747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c84f84-fc37-45fc-9f9d-82e23c11c7c3 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:24.531684 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:24.545213 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c84f84-fc37-45fc-9f9d-82e23c11c7c3 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:24.549758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:24.550378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:24.550654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:24.550920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] PATH_INFO: `/v3/users/a2c8189bb1254baf93c2353cea6f5b8d` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:24.552021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c8189bb1254baf93c2353cea6f5b8d)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:24.554138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:24.599728 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c8189bb1254baf93c2353cea6f5b8d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:24.601111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c8189bb1254baf93c2353cea6f5b8d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:24.613475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c8189bb1254baf93c2353cea6f5b8d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:24.620970 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c8189bb1254baf93c2353cea6f5b8d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:24.628316 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c8189bb1254baf93c2353cea6f5b8d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:24.634696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1378/2780] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:24 2022] DELETE /identity/v3/users/a2c8189bb1254baf93c2353cea6f5b8d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:38:24.639571 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8b73b-567d-465f-99a0-40c995d59fad None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:24.640073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8b73b-567d-465f-99a0-40c995d59fad None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:24.640210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8b73b-567d-465f-99a0-40c995d59fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:24.660159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-97b8b73b-567d-465f-99a0-40c995d59fad None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:24.661501 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-97b8b73b-567d-465f-99a0-40c995d59fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:24.725795 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1403/2781] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:24.735569 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:24.736999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:24.796306 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd4b28-4200-44ae-8c34-0f377d2e77ba admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:24.799952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421535f3-00a2-4cf4-aa3a-a2dae4ab31fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:24.800587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421535f3-00a2-4cf4-aa3a-a2dae4ab31fc service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:24.800866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421535f3-00a2-4cf4-aa3a-a2dae4ab31fc service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:24.801136 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421535f3-00a2-4cf4-aa3a-a2dae4ab31fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:24.810208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421535f3-00a2-4cf4-aa3a-a2dae4ab31fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:24.811645 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421535f3-00a2-4cf4-aa3a-a2dae4ab31fc service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:24.821699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1379/2782] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:25.098452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8b73b-567d-465f-99a0-40c995d59fad None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:25.100203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-97b8b73b-567d-465f-99a0-40c995d59fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:25.118603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8b73b-567d-465f-99a0-40c995d59fad None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:25.123839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:25.124474 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.124776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.125044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] PATH_INFO: `/v3/projects/0bbc14c088064941bb7d6104305aa82b` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.126231 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bbc14c088064941bb7d6104305aa82b)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:25.128217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:25.182340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bbc14c088064941bb7d6104305aa82b', 'request_id': 'req-13939764-8f73-4d7c-b03f-518ecf1e2abf'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:25.188650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:25.190017 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bbc14c088064941bb7d6104305aa82b', 'request_id': 'req-13939764-8f73-4d7c-b03f-518ecf1e2abf'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:25.197921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bbc14c088064941bb7d6104305aa82b', 'request_id': 'req-13939764-8f73-4d7c-b03f-518ecf1e2abf'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:25.211669 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] The token cache is being invalidate because project 0bbc14c088064941bb7d6104305aa82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:25.211991 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:25.212670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-13939764-8f73-4d7c-b03f-518ecf1e2abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:25.214161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1404/2783] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:25 2022] DELETE /identity/v3/projects/0bbc14c088064941bb7d6104305aa82b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:25.219155 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99c9b6-f90a-4104-8c74-1be122d36c5b None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.219439 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99c9b6-f90a-4104-8c74-1be122d36c5b None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.219737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99c9b6-f90a-4104-8c74-1be122d36c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.241784 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-ca99c9b6-f90a-4104-8c74-1be122d36c5b None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:25.242978 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ca99c9b6-f90a-4104-8c74-1be122d36c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:25.315223 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1380/2784] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:25.323797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aac063-0690-4a8b-bfe5-1ab6647d49cb None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.324195 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aac063-0690-4a8b-bfe5-1ab6647d49cb None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.324574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aac063-0690-4a8b-bfe5-1ab6647d49cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.344665 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-64aac063-0690-4a8b-bfe5-1ab6647d49cb None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:25.345880 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-64aac063-0690-4a8b-bfe5-1ab6647d49cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:25.354457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1405/2785] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:25.358962 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99c9b6-f90a-4104-8c74-1be122d36c5b None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:25.370623 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99c9b6-f90a-4104-8c74-1be122d36c5b None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:25.375529 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4fd58-9607-46ca-982b-6aa77374fa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:25.376138 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4fd58-9607-46ca-982b-6aa77374fa79 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.376445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4fd58-9607-46ca-982b-6aa77374fa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.376748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4fd58-9607-46ca-982b-6aa77374fa79 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.377399 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4fd58-9607-46ca-982b-6aa77374fa79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:38:25.377701 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4fd58-9607-46ca-982b-6aa77374fa79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:25.379300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4fd58-9607-46ca-982b-6aa77374fa79 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:25.399764 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1381/2786] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:38:25.405483 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aac063-0690-4a8b-bfe5-1ab6647d49cb None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:25.415105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aac063-0690-4a8b-bfe5-1ab6647d49cb None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:25.419914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395de5d7-d2b6-4112-82cb-a1ff9ebb3732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:25.420552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395de5d7-d2b6-4112-82cb-a1ff9ebb3732 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.420966 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395de5d7-d2b6-4112-82cb-a1ff9ebb3732 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.421149 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395de5d7-d2b6-4112-82cb-a1ff9ebb3732 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.421888 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395de5d7-d2b6-4112-82cb-a1ff9ebb3732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:25.423942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395de5d7-d2b6-4112-82cb-a1ff9ebb3732 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:25.453767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1406/2787] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:25 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:25.457991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4fd58-9607-46ca-982b-6aa77374fa79 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:25.463932 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:25.470615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4fd58-9607-46ca-982b-6aa77374fa79 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:25.532424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e17ad6-0db9-4ca7-a19e-301a5461fb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:25.533211 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e17ad6-0db9-4ca7-a19e-301a5461fb28 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.533596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e17ad6-0db9-4ca7-a19e-301a5461fb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.534005 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e17ad6-0db9-4ca7-a19e-301a5461fb28 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.534820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e17ad6-0db9-4ca7-a19e-301a5461fb28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:25.537044 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e17ad6-0db9-4ca7-a19e-301a5461fb28 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:25.564895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1382/2788] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:25 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:25.569902 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395de5d7-d2b6-4112-82cb-a1ff9ebb3732 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:25.580979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395de5d7-d2b6-4112-82cb-a1ff9ebb3732 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:25.589954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f10061-e3cf-4765-aeaa-3d3cb9220dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:25.590675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f10061-e3cf-4765-aeaa-3d3cb9220dff admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.590988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f10061-e3cf-4765-aeaa-3d3cb9220dff admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.591332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f10061-e3cf-4765-aeaa-3d3cb9220dff admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.592053 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f10061-e3cf-4765-aeaa-3d3cb9220dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:25.594247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f10061-e3cf-4765-aeaa-3d3cb9220dff admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:25.610637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1407/2789] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:25.612803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e17ad6-0db9-4ca7-a19e-301a5461fb28 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:25.623267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e17ad6-0db9-4ca7-a19e-301a5461fb28 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:25.628068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57db12c-2efd-4c18-8103-861768b9de1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:25.628695 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57db12c-2efd-4c18-8103-861768b9de1c admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.628971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57db12c-2efd-4c18-8103-861768b9de1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.629237 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57db12c-2efd-4c18-8103-861768b9de1c admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.630104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57db12c-2efd-4c18-8103-861768b9de1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:25.633053 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57db12c-2efd-4c18-8103-861768b9de1c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:25.648553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1383/2790] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:25.650356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f10061-e3cf-4765-aeaa-3d3cb9220dff admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:25.668545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f10061-e3cf-4765-aeaa-3d3cb9220dff admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:25.668545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa3bd-6070-4cbb-8d86-9e1a15e68412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:25.668545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa3bd-6070-4cbb-8d86-9e1a15e68412 admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.668545 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa3bd-6070-4cbb-8d86-9e1a15e68412 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.669032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3faa3bd-6070-4cbb-8d86-9e1a15e68412 admin admin] PATH_INFO: `/v3/projects/cf8a0aad2b0246ec9d5134edbd5dfe78/users/d276b214f28c4397b39ac50756319367/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.685210 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faa3bd-6070-4cbb-8d86-9e1a15e68412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8a0aad2b0246ec9d5134edbd5dfe78, user_id=d276b214f28c4397b39ac50756319367, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:25.687872 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3faa3bd-6070-4cbb-8d86-9e1a15e68412 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:25.701232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1408/2791] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:25 2022] PUT /identity/v3/projects/cf8a0aad2b0246ec9d5134edbd5dfe78/users/d276b214f28c4397b39ac50756319367/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:25.712020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd1421-1ebc-4e76-92e3-c1f60565d7be None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.712304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd1421-1ebc-4e76-92e3-c1f60565d7be None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.712617 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd1421-1ebc-4e76-92e3-c1f60565d7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.743402 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2fcd1421-1ebc-4e76-92e3-c1f60565d7be None None] MFA Rules not processed for user `d276b214f28c4397b39ac50756319367`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:25.744876 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2fcd1421-1ebc-4e76-92e3-c1f60565d7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:25.816346 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1384/2792] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:25 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:25.828773 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa3bd-6070-4cbb-8d86-9e1a15e68412 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:25.830535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b3faa3bd-6070-4cbb-8d86-9e1a15e68412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:25.898549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3faa3bd-6070-4cbb-8d86-9e1a15e68412 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:25.902229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e0b8b-11ec-4bad-bddd-5c94dd611f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:25.902837 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e0b8b-11ec-4bad-bddd-5c94dd611f1e service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:25.903111 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e0b8b-11ec-4bad-bddd-5c94dd611f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:25.903381 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e0b8b-11ec-4bad-bddd-5c94dd611f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:25.912671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e0b8b-11ec-4bad-bddd-5c94dd611f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:25.914806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e0b8b-11ec-4bad-bddd-5c94dd611f1e service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:25.924815 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1409/2793] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:27.832968 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd1421-1ebc-4e76-92e3-c1f60565d7be None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:27.835378 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2fcd1421-1ebc-4e76-92e3-c1f60565d7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:27.906727 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd1421-1ebc-4e76-92e3-c1f60565d7be None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:27.911396 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:27.912028 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:27.912300 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:27.913233 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] PATH_INFO: `/v3/users/9c650f95287048eda94fb368520c5686` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:27.914426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c650f95287048eda94fb368520c5686)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:27.916352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:27.931530 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:27.964789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c650f95287048eda94fb368520c5686'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:27.965442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c650f95287048eda94fb368520c5686'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:27.977707 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c650f95287048eda94fb368520c5686'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:27.984522 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c650f95287048eda94fb368520c5686'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:27.992509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c650f95287048eda94fb368520c5686'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:27.998459 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1385/2794] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:27 2022] DELETE /identity/v3/users/9c650f95287048eda94fb368520c5686 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:28.003088 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e86ff-6f2e-4d0a-b198-925ac86462cc None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:28.003564 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e86ff-6f2e-4d0a-b198-925ac86462cc None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:28.003673 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e86ff-6f2e-4d0a-b198-925ac86462cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:28.022514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-fd3e86ff-6f2e-4d0a-b198-925ac86462cc None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:28.023705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fd3e86ff-6f2e-4d0a-b198-925ac86462cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:28.082006 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1410/2795] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:28.092087 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:28.093478 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:28.151058 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a051c13-0158-42cc-99d3-b312da60bacf admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:28.154687 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f83e7-c3b6-4b9c-8113-4590f046f55a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:28.155289 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f83e7-c3b6-4b9c-8113-4590f046f55a service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:28.155565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f83e7-c3b6-4b9c-8113-4590f046f55a service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:28.155834 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f83e7-c3b6-4b9c-8113-4590f046f55a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:28.165603 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f83e7-c3b6-4b9c-8113-4590f046f55a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:28.167035 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f83e7-c3b6-4b9c-8113-4590f046f55a service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:28.176734 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1386/2796] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:28.436487 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e86ff-6f2e-4d0a-b198-925ac86462cc None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:28.437977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-fd3e86ff-6f2e-4d0a-b198-925ac86462cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:28.451204 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e86ff-6f2e-4d0a-b198-925ac86462cc None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:28.456409 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:28.456716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:28.456937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:28.457213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] PATH_INFO: `/v3/projects/0814baf2ffe348b98cd011544df83c0e` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:28.458368 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0814baf2ffe348b98cd011544df83c0e)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:28.461380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:28.485823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:28.519926 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0814baf2ffe348b98cd011544df83c0e', 'request_id': 'req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.527942 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0814baf2ffe348b98cd011544df83c0e', 'request_id': 'req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.535245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0814baf2ffe348b98cd011544df83c0e', 'request_id': 'req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.549247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] The token cache is being invalidate because project 0814baf2ffe348b98cd011544df83c0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:28.549247 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.549732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.551350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1411/2797] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:28 2022] DELETE /identity/v3/projects/0814baf2ffe348b98cd011544df83c0e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:28.555853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f83e7-c3b6-4b9c-8113-4590f046f55a service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:28.565542 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f83e7-c3b6-4b9c-8113-4590f046f55a service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:28.627148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:28.627781 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:28.628059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:28.628353 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] PATH_INFO: `/v3/users/2a330260610b4f0396e13a8dd0660b78` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:28.629512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a330260610b4f0396e13a8dd0660b78)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:28.632426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:28.672322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:28.682471 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a330260610b4f0396e13a8dd0660b78'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.683540 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a330260610b4f0396e13a8dd0660b78'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.696633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a330260610b4f0396e13a8dd0660b78'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.703694 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a330260610b4f0396e13a8dd0660b78'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.712878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-49a6aced-7c4c-4800-8d28-2097218670f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a330260610b4f0396e13a8dd0660b78'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:28.718899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1387/2798] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:28 2022] DELETE /identity/v3/users/2a330260610b4f0396e13a8dd0660b78 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:29.003892 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:29.005356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:29.019453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564ebe3-3ea1-4b48-9e40-b4cfd8586774 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:29.078011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:29.078639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.078916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.079189 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] PATH_INFO: `/v3/projects/81fc1692609642c6ae36457f5da656d2` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.080322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81fc1692609642c6ae36457f5da656d2)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:29.082280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:29.127981 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81fc1692609642c6ae36457f5da656d2', 'request_id': 'req-699d0446-e1c3-47b5-921c-30d17db451b8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:29.134429 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81fc1692609642c6ae36457f5da656d2', 'request_id': 'req-699d0446-e1c3-47b5-921c-30d17db451b8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:29.140560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81fc1692609642c6ae36457f5da656d2', 'request_id': 'req-699d0446-e1c3-47b5-921c-30d17db451b8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:29.152833 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] The token cache is being invalidate because project 81fc1692609642c6ae36457f5da656d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:29.153124 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:29.153753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-699d0446-e1c3-47b5-921c-30d17db451b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:29.155228 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1412/2799] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:29 2022] DELETE /identity/v3/projects/81fc1692609642c6ae36457f5da656d2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:29.161168 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70fbf4-7176-4620-98d2-55da048094cb None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.161650 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70fbf4-7176-4620-98d2-55da048094cb None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.161749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70fbf4-7176-4620-98d2-55da048094cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.181999 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2e70fbf4-7176-4620-98d2-55da048094cb None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:29.183188 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2e70fbf4-7176-4620-98d2-55da048094cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:29.241541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1388/2800] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:29.249384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454c415-0ed8-4f6e-a88c-50f693420e20 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.249667 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454c415-0ed8-4f6e-a88c-50f693420e20 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.249941 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454c415-0ed8-4f6e-a88c-50f693420e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.269415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3454c415-0ed8-4f6e-a88c-50f693420e20 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:29.270603 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3454c415-0ed8-4f6e-a88c-50f693420e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:29.278317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1413/2801] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:29.282433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70fbf4-7176-4620-98d2-55da048094cb None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:29.292776 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70fbf4-7176-4620-98d2-55da048094cb None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:29.297279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7527-c074-40d0-bcd2-69706dbf62ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:29.297871 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7527-c074-40d0-bcd2-69706dbf62ab admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.298145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7527-c074-40d0-bcd2-69706dbf62ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.298413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7527-c074-40d0-bcd2-69706dbf62ab admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.299062 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7527-c074-40d0-bcd2-69706dbf62ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:38:29.299360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7527-c074-40d0-bcd2-69706dbf62ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:29.300986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7527-c074-40d0-bcd2-69706dbf62ab admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:29.313068 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1389/2802] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:38:29.317212 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454c415-0ed8-4f6e-a88c-50f693420e20 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:29.326679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454c415-0ed8-4f6e-a88c-50f693420e20 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:29.331344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc2268-e5d9-4755-8347-8048e052398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:29.332159 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc2268-e5d9-4755-8347-8048e052398c admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.332434 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc2268-e5d9-4755-8347-8048e052398c admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.332744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc2268-e5d9-4755-8347-8048e052398c admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.333353 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc2268-e5d9-4755-8347-8048e052398c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:29.335435 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc2268-e5d9-4755-8347-8048e052398c admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:29.362989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1414/2803] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:29 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:29.367714 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7527-c074-40d0-bcd2-69706dbf62ab admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:29.377761 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7527-c074-40d0-bcd2-69706dbf62ab admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:29.433480 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046606a5-1eed-4645-883d-8c06b710de08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:29.434659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046606a5-1eed-4645-883d-8c06b710de08 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.434659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046606a5-1eed-4645-883d-8c06b710de08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.434659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046606a5-1eed-4645-883d-8c06b710de08 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.435252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046606a5-1eed-4645-883d-8c06b710de08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:29.437199 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046606a5-1eed-4645-883d-8c06b710de08 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:29.458681 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1390/2804] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:29 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:29.462907 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc2268-e5d9-4755-8347-8048e052398c admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:29.473140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc2268-e5d9-4755-8347-8048e052398c admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:29.478150 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14798a-8712-4a45-ad83-1896e51550ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:29.478732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14798a-8712-4a45-ad83-1896e51550ad admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.479005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14798a-8712-4a45-ad83-1896e51550ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.479272 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14798a-8712-4a45-ad83-1896e51550ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.479897 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14798a-8712-4a45-ad83-1896e51550ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:29.481519 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14798a-8712-4a45-ad83-1896e51550ad admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:29.482749 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:29.493585 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1415/2805] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:38:29.497263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046606a5-1eed-4645-883d-8c06b710de08 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:29.507367 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046606a5-1eed-4645-883d-8c06b710de08 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:29.511934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d3b4d-65c5-4dae-bf72-f88a6f55ec05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:29.512585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d3b4d-65c5-4dae-bf72-f88a6f55ec05 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.512866 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d3b4d-65c5-4dae-bf72-f88a6f55ec05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.513139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d3b4d-65c5-4dae-bf72-f88a6f55ec05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.513779 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d3b4d-65c5-4dae-bf72-f88a6f55ec05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:29.515372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d3b4d-65c5-4dae-bf72-f88a6f55ec05 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:29.526732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1391/2806] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:29.530899 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14798a-8712-4a45-ad83-1896e51550ad admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:29.541579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14798a-8712-4a45-ad83-1896e51550ad admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:29.546404 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08582eb6-df77-4d87-bea3-e5ce7ab721bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:29.547102 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08582eb6-df77-4d87-bea3-e5ce7ab721bb admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.547382 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08582eb6-df77-4d87-bea3-e5ce7ab721bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.547655 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08582eb6-df77-4d87-bea3-e5ce7ab721bb admin admin] PATH_INFO: `/v3/projects/9d3eced9b3c3485db7e492adfe7f7553/users/d0e0edd3842742f4bb8254cbe1012976/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.562754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08582eb6-df77-4d87-bea3-e5ce7ab721bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3eced9b3c3485db7e492adfe7f7553, user_id=d0e0edd3842742f4bb8254cbe1012976, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:29.565453 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08582eb6-df77-4d87-bea3-e5ce7ab721bb admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:29.578491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1416/2807] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:29 2022] PUT /identity/v3/projects/9d3eced9b3c3485db7e492adfe7f7553/users/d0e0edd3842742f4bb8254cbe1012976/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:29.591240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd3e04-7828-487d-8026-110ff9489b00 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.591523 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd3e04-7828-487d-8026-110ff9489b00 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.591792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd3e04-7828-487d-8026-110ff9489b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.619009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-8fdd3e04-7828-487d-8026-110ff9489b00 None None] MFA Rules not processed for user `d0e0edd3842742f4bb8254cbe1012976`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:29.620194 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8fdd3e04-7828-487d-8026-110ff9489b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:29.625065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:29.687250 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1392/2808] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:29.699943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08582eb6-df77-4d87-bea3-e5ce7ab721bb admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:29.701690 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-08582eb6-df77-4d87-bea3-e5ce7ab721bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:29.763732 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08582eb6-df77-4d87-bea3-e5ce7ab721bb admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:29.767592 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70a2d9-4bbd-49eb-afb5-2d8d405d108b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:29.768226 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70a2d9-4bbd-49eb-afb5-2d8d405d108b service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:29.768531 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70a2d9-4bbd-49eb-afb5-2d8d405d108b service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:29.768809 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70a2d9-4bbd-49eb-afb5-2d8d405d108b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:29.777724 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70a2d9-4bbd-49eb-afb5-2d8d405d108b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:29.779180 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70a2d9-4bbd-49eb-afb5-2d8d405d108b service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:29.788886 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1417/2809] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:36.132238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd3e04-7828-487d-8026-110ff9489b00 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:36.133665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-8fdd3e04-7828-487d-8026-110ff9489b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:36.198096 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd3e04-7828-487d-8026-110ff9489b00 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:36.202640 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:36.203264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:36.203536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:36.203830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] PATH_INFO: `/v3/users/d276b214f28c4397b39ac50756319367` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:36.204965 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d276b214f28c4397b39ac50756319367)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:36.206874 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:36.256282 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd276b214f28c4397b39ac50756319367'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.257487 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd276b214f28c4397b39ac50756319367'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.274046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd276b214f28c4397b39ac50756319367'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.280794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd276b214f28c4397b39ac50756319367'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.288933 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd276b214f28c4397b39ac50756319367'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.294721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1393/2810] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:36 2022] DELETE /identity/v3/users/d276b214f28c4397b39ac50756319367 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:36.299283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec07a5a-4476-458e-8fc9-5046058811dd None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:36.299766 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec07a5a-4476-458e-8fc9-5046058811dd None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:36.300273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec07a5a-4476-458e-8fc9-5046058811dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:36.318871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-cec07a5a-4476-458e-8fc9-5046058811dd None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:36.320705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-cec07a5a-4476-458e-8fc9-5046058811dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:36.378538 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1418/2811] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:36.388964 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:36.390398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:36.420619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:36.448986 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b70b2b-7a15-471e-af2d-d38ac3d86179 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:36.452608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be400d-e8ce-443e-85bc-29f43527d7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:36.453205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be400d-e8ce-443e-85bc-29f43527d7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:36.453479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be400d-e8ce-443e-85bc-29f43527d7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:36.453744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be400d-e8ce-443e-85bc-29f43527d7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:36.462571 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be400d-e8ce-443e-85bc-29f43527d7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:36.464090 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be400d-e8ce-443e-85bc-29f43527d7a9 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:36.473332 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1394/2812] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:36.739998 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec07a5a-4476-458e-8fc9-5046058811dd None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:36.742527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-cec07a5a-4476-458e-8fc9-5046058811dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:36.770160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec07a5a-4476-458e-8fc9-5046058811dd None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:36.776466 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:36.777135 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:36.777412 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:36.777679 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] PATH_INFO: `/v3/projects/cf8a0aad2b0246ec9d5134edbd5dfe78` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:36.778798 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf8a0aad2b0246ec9d5134edbd5dfe78)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:36.781264 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:36.832829 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf8a0aad2b0246ec9d5134edbd5dfe78', 'request_id': 'req-354f383c-9ed2-445d-b737-7f3880afa753'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.838702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf8a0aad2b0246ec9d5134edbd5dfe78', 'request_id': 'req-354f383c-9ed2-445d-b737-7f3880afa753'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.845442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf8a0aad2b0246ec9d5134edbd5dfe78', 'request_id': 'req-354f383c-9ed2-445d-b737-7f3880afa753'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.854965 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:36.860282 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] The token cache is being invalidate because project cf8a0aad2b0246ec9d5134edbd5dfe78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:36.860840 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.861725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-354f383c-9ed2-445d-b737-7f3880afa753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:36.863796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1419/2813] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:36 2022] DELETE /identity/v3/projects/cf8a0aad2b0246ec9d5134edbd5dfe78 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:46.469217 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=14 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:46.872871 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:52.630351 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be400d-e8ce-443e-85bc-29f43527d7a9 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:52.632360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-a9be400d-e8ce-443e-85bc-29f43527d7a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:52.644921 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be400d-e8ce-443e-85bc-29f43527d7a9 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:52.701895 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:52.702519 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:52.702793 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:52.703061 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] PATH_INFO: `/v3/users/d0e0edd3842742f4bb8254cbe1012976` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:52.704175 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e0edd3842742f4bb8254cbe1012976)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:52.706122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:52.752329 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e0edd3842742f4bb8254cbe1012976'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:52.752922 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e0edd3842742f4bb8254cbe1012976'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:52.766747 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e0edd3842742f4bb8254cbe1012976'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:52.775031 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e0edd3842742f4bb8254cbe1012976'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:52.781882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e0edd3842742f4bb8254cbe1012976'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:52.788081 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1395/2814] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:38:52 2022] DELETE /identity/v3/users/d0e0edd3842742f4bb8254cbe1012976 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:52.792470 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9fb95-b106-4ac0-abd8-7a11a96f2bda None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:52.792976 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9fb95-b106-4ac0-abd8-7a11a96f2bda None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:52.793096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9fb95-b106-4ac0-abd8-7a11a96f2bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:52.812574 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-1ca9fb95-b106-4ac0-abd8-7a11a96f2bda None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:52.813771 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1ca9fb95-b106-4ac0-abd8-7a11a96f2bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:52.872320 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1420/2815] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:52.883698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:52.885148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:52.947619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e2f2e-8173-44c3-bf96-c1baa1feed9a admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:52.951222 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe3765-f5d2-4296-8dee-9e36afe502b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:52.951826 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3765-f5d2-4296-8dee-9e36afe502b3 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:52.952412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3765-f5d2-4296-8dee-9e36afe502b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:52.952704 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3765-f5d2-4296-8dee-9e36afe502b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:52.961631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe3765-f5d2-4296-8dee-9e36afe502b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:52.963063 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe3765-f5d2-4296-8dee-9e36afe502b3 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:52.973187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1396/2816] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:53.238835 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9fb95-b106-4ac0-abd8-7a11a96f2bda None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:53.240441 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-1ca9fb95-b106-4ac0-abd8-7a11a96f2bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:53.254522 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9fb95-b106-4ac0-abd8-7a11a96f2bda None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:53.260682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:53.261313 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.261591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.261864 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] PATH_INFO: `/v3/projects/9d3eced9b3c3485db7e492adfe7f7553` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.263075 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3eced9b3c3485db7e492adfe7f7553)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:53.265286 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:53.312356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3eced9b3c3485db7e492adfe7f7553', 'request_id': 'req-5b7da7d6-dd5d-4822-9369-789521993631'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:53.318040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3eced9b3c3485db7e492adfe7f7553', 'request_id': 'req-5b7da7d6-dd5d-4822-9369-789521993631'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:53.325019 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3eced9b3c3485db7e492adfe7f7553', 'request_id': 'req-5b7da7d6-dd5d-4822-9369-789521993631'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:53.337612 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] The token cache is being invalidate because project 9d3eced9b3c3485db7e492adfe7f7553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:38:53.337898 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:53.338507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-5b7da7d6-dd5d-4822-9369-789521993631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:38:53.339988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1421/2817] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:38:53 2022] DELETE /identity/v3/projects/9d3eced9b3c3485db7e492adfe7f7553 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:53.344931 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a960e3f-c527-4c4b-88ab-67c7f3e81e5e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.345218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a960e3f-c527-4c4b-88ab-67c7f3e81e5e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.345491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a960e3f-c527-4c4b-88ab-67c7f3e81e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.365111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-7a960e3f-c527-4c4b-88ab-67c7f3e81e5e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:53.366368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-7a960e3f-c527-4c4b-88ab-67c7f3e81e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:53.383544 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:53.427206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1397/2818] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:53.435221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8037ee0-ff36-42bb-af48-e7955cae5299 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.435573 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8037ee0-ff36-42bb-af48-e7955cae5299 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.435850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8037ee0-ff36-42bb-af48-e7955cae5299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.455694 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-f8037ee0-ff36-42bb-af48-e7955cae5299 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:53.456920 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f8037ee0-ff36-42bb-af48-e7955cae5299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:53.465294 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1422/2819] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:53.469481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a960e3f-c527-4c4b-88ab-67c7f3e81e5e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:53.479059 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a960e3f-c527-4c4b-88ab-67c7f3e81e5e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:53.483769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d004fff-7584-4f18-be8e-abb6fb7814b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:53.484361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d004fff-7584-4f18-be8e-abb6fb7814b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.484669 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d004fff-7584-4f18-be8e-abb6fb7814b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.484941 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d004fff-7584-4f18-be8e-abb6fb7814b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.485583 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d004fff-7584-4f18-be8e-abb6fb7814b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93106) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:38:53.485908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d004fff-7584-4f18-be8e-abb6fb7814b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:53.487506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d004fff-7584-4f18-be8e-abb6fb7814b6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:53.499319 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1398/2820] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:38:53.503393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8037ee0-ff36-42bb-af48-e7955cae5299 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:53.512914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8037ee0-ff36-42bb-af48-e7955cae5299 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:53.517421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024b87d-841c-40e2-a6f1-94b6a46f0980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:53.518030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024b87d-841c-40e2-a6f1-94b6a46f0980 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.518305 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024b87d-841c-40e2-a6f1-94b6a46f0980 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.518591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024b87d-841c-40e2-a6f1-94b6a46f0980 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.519217 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024b87d-841c-40e2-a6f1-94b6a46f0980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:53.521155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024b87d-841c-40e2-a6f1-94b6a46f0980 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:53.548164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1423/2821] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:53 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:53.552456 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d004fff-7584-4f18-be8e-abb6fb7814b6 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:53.563225 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d004fff-7584-4f18-be8e-abb6fb7814b6 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:53.622074 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82644d80-3941-4ada-be3e-072c5d82e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:53.622678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82644d80-3941-4ada-be3e-072c5d82e9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.622953 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82644d80-3941-4ada-be3e-072c5d82e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.623221 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82644d80-3941-4ada-be3e-072c5d82e9d2 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.623834 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82644d80-3941-4ada-be3e-072c5d82e9d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:53.625997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82644d80-3941-4ada-be3e-072c5d82e9d2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:53.647333 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1399/2822] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:53 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:53.651465 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024b87d-841c-40e2-a6f1-94b6a46f0980 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:53.661816 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024b87d-841c-40e2-a6f1-94b6a46f0980 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:53.667085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071c3b5-6cfc-4846-a737-b84b75686e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:53.667699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071c3b5-6cfc-4846-a737-b84b75686e29 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.667967 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071c3b5-6cfc-4846-a737-b84b75686e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.668241 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071c3b5-6cfc-4846-a737-b84b75686e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.668921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071c3b5-6cfc-4846-a737-b84b75686e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:53.670527 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071c3b5-6cfc-4846-a737-b84b75686e29 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:53.681418 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1424/2823] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:53.685272 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82644d80-3941-4ada-be3e-072c5d82e9d2 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:53.695676 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82644d80-3941-4ada-be3e-072c5d82e9d2 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:53.700484 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086b186-8fdb-44ce-9351-d98ea2b440cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:53.701104 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086b186-8fdb-44ce-9351-d98ea2b440cc admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.701376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086b186-8fdb-44ce-9351-d98ea2b440cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.701642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086b186-8fdb-44ce-9351-d98ea2b440cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.702274 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086b186-8fdb-44ce-9351-d98ea2b440cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:53.703857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086b186-8fdb-44ce-9351-d98ea2b440cc admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:53.714590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1400/2824] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:53.718651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071c3b5-6cfc-4846-a737-b84b75686e29 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:53.728520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071c3b5-6cfc-4846-a737-b84b75686e29 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:53.733051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd738b-16f5-4914-8a90-6e23da1cc3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:53.733713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd738b-16f5-4914-8a90-6e23da1cc3ec admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.733986 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd738b-16f5-4914-8a90-6e23da1cc3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.734255 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd738b-16f5-4914-8a90-6e23da1cc3ec admin admin] PATH_INFO: `/v3/projects/c83af4d1c60a4d00a5a8ed1b28dd3294/users/fca8cb665f5841609a5e271cbd100003/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.748989 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd738b-16f5-4914-8a90-6e23da1cc3ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83af4d1c60a4d00a5a8ed1b28dd3294, user_id=fca8cb665f5841609a5e271cbd100003, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:53.751617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd738b-16f5-4914-8a90-6e23da1cc3ec admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:53.765232 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1425/2825] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:53 2022] PUT /identity/v3/projects/c83af4d1c60a4d00a5a8ed1b28dd3294/users/fca8cb665f5841609a5e271cbd100003/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:53.775252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2040bd-d8ff-4a07-85ba-aef157efd7ec None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.775566 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2040bd-d8ff-4a07-85ba-aef157efd7ec None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.775948 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2040bd-d8ff-4a07-85ba-aef157efd7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.803945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-df2040bd-d8ff-4a07-85ba-aef157efd7ec None None] MFA Rules not processed for user `fca8cb665f5841609a5e271cbd100003`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:53.805193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-df2040bd-d8ff-4a07-85ba-aef157efd7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:53.868462 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1401/2826] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:53.872357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd738b-16f5-4914-8a90-6e23da1cc3ec admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:53.893497 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:53.937222 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd738b-16f5-4914-8a90-6e23da1cc3ec admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:53.941830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae5adf-8743-404b-9f13-46bbaf206446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:53.942426 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae5adf-8743-404b-9f13-46bbaf206446 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:53.942704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae5adf-8743-404b-9f13-46bbaf206446 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:53.942977 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae5adf-8743-404b-9f13-46bbaf206446 admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:53.943716 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae5adf-8743-404b-9f13-46bbaf206446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:53.945661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae5adf-8743-404b-9f13-46bbaf206446 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:53.971681 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1426/2827] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:53 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:53.976444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2040bd-d8ff-4a07-85ba-aef157efd7ec None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:53.987928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2040bd-d8ff-4a07-85ba-aef157efd7ec None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.000771 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:54.045827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febc778-6e0d-4bf6-ae44-7dfe122d8bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.046418 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febc778-6e0d-4bf6-ae44-7dfe122d8bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.046698 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febc778-6e0d-4bf6-ae44-7dfe122d8bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.046971 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febc778-6e0d-4bf6-ae44-7dfe122d8bf2 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.047639 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febc778-6e0d-4bf6-ae44-7dfe122d8bf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.049608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febc778-6e0d-4bf6-ae44-7dfe122d8bf2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.073881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1402/2828] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:53 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:54.077985 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae5adf-8743-404b-9f13-46bbaf206446 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.088151 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae5adf-8743-404b-9f13-46bbaf206446 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.092764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423f368-3989-4b77-8005-e123ed806550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.093364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423f368-3989-4b77-8005-e123ed806550 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.093645 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423f368-3989-4b77-8005-e123ed806550 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.093918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423f368-3989-4b77-8005-e123ed806550 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.094556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423f368-3989-4b77-8005-e123ed806550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.096196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423f368-3989-4b77-8005-e123ed806550 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.107413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1427/2829] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:54.111413 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febc778-6e0d-4bf6-ae44-7dfe122d8bf2 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.122201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febc778-6e0d-4bf6-ae44-7dfe122d8bf2 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.127475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0d079-d413-43ff-b648-86fce8201249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.128070 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d079-d413-43ff-b648-86fce8201249 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.128352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d079-d413-43ff-b648-86fce8201249 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.128665 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0d079-d413-43ff-b648-86fce8201249 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.129308 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d079-d413-43ff-b648-86fce8201249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.130928 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0d079-d413-43ff-b648-86fce8201249 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.141137 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1403/2830] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:54.145140 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423f368-3989-4b77-8005-e123ed806550 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.155040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423f368-3989-4b77-8005-e123ed806550 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.159682 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ede68-41ad-4667-82e4-4c4195f4c98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.160396 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ede68-41ad-4667-82e4-4c4195f4c98f admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.160764 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ede68-41ad-4667-82e4-4c4195f4c98f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.161039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ede68-41ad-4667-82e4-4c4195f4c98f admin admin] PATH_INFO: `/v3/projects/e996df60c24c4f7cb166d3d17ccf41b3/users/ac1dd6b56cfe4814a1583cd2ad43c071/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.175928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ede68-41ad-4667-82e4-4c4195f4c98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e996df60c24c4f7cb166d3d17ccf41b3, user_id=ac1dd6b56cfe4814a1583cd2ad43c071, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.178737 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ede68-41ad-4667-82e4-4c4195f4c98f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.191174 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1428/2831] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:54 2022] PUT /identity/v3/projects/e996df60c24c4f7cb166d3d17ccf41b3/users/ac1dd6b56cfe4814a1583cd2ad43c071/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:54.203252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf922e-5675-47df-821b-467aa395584f None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.203538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf922e-5675-47df-821b-467aa395584f None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.203812 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf922e-5675-47df-821b-467aa395584f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.231981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2daf922e-5675-47df-821b-467aa395584f None None] MFA Rules not processed for user `ac1dd6b56cfe4814a1583cd2ad43c071`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:54.233924 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2daf922e-5675-47df-821b-467aa395584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:54.299803 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1404/2832] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:54.305003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ede68-41ad-4667-82e4-4c4195f4c98f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.368972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ede68-41ad-4667-82e4-4c4195f4c98f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.373636 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2fd76-25b0-4070-965a-8b4c91c81b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.374191 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2fd76-25b0-4070-965a-8b4c91c81b2e admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.374495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2fd76-25b0-4070-965a-8b4c91c81b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.374769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2fd76-25b0-4070-965a-8b4c91c81b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.375394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2fd76-25b0-4070-965a-8b4c91c81b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.377339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2fd76-25b0-4070-965a-8b4c91c81b2e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.409273 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1429/2833] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:38:54 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:38:54.413264 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf922e-5675-47df-821b-467aa395584f None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.423950 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf922e-5675-47df-821b-467aa395584f None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.481835 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73cd65-606d-44ce-bfdb-7d9f91395af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.482442 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73cd65-606d-44ce-bfdb-7d9f91395af6 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.482718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73cd65-606d-44ce-bfdb-7d9f91395af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.482992 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73cd65-606d-44ce-bfdb-7d9f91395af6 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.483609 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73cd65-606d-44ce-bfdb-7d9f91395af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.485574 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73cd65-606d-44ce-bfdb-7d9f91395af6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.507271 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1405/2834] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:38:54 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:38:54.511421 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2fd76-25b0-4070-965a-8b4c91c81b2e admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.521581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2fd76-25b0-4070-965a-8b4c91c81b2e admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.526238 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92988611-bade-4013-b7d6-3c9b05867a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.526828 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92988611-bade-4013-b7d6-3c9b05867a6d admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.527104 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92988611-bade-4013-b7d6-3c9b05867a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.527372 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92988611-bade-4013-b7d6-3c9b05867a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.528003 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92988611-bade-4013-b7d6-3c9b05867a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.529657 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92988611-bade-4013-b7d6-3c9b05867a6d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.540216 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1430/2835] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:54.544260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73cd65-606d-44ce-bfdb-7d9f91395af6 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.554829 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73cd65-606d-44ce-bfdb-7d9f91395af6 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.559383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f61a66-0f42-4184-9fb4-4a22ee572b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.560025 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61a66-0f42-4184-9fb4-4a22ee572b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.560304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61a66-0f42-4184-9fb4-4a22ee572b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.560642 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61a66-0f42-4184-9fb4-4a22ee572b5d admin admin] PATH_INFO: `/v3/projects/41096bb76c3b47be9568d38a9f52766a/users/a48d0f7b67804548a9d047a12a806a84/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.573820 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:54.577197 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f61a66-0f42-4184-9fb4-4a22ee572b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=41096bb76c3b47be9568d38a9f52766a, user_id=a48d0f7b67804548a9d047a12a806a84, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.580445 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f61a66-0f42-4184-9fb4-4a22ee572b5d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.593506 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1406/2836] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:54 2022] PUT /identity/v3/projects/41096bb76c3b47be9568d38a9f52766a/users/a48d0f7b67804548a9d047a12a806a84/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:54.597374 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92988611-bade-4013-b7d6-3c9b05867a6d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.607579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92988611-bade-4013-b7d6-3c9b05867a6d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.650999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:38:54.668689 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152a8c3-7a51-415a-b628-8634617eab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.669299 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152a8c3-7a51-415a-b628-8634617eab10 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.669576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152a8c3-7a51-415a-b628-8634617eab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.669846 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152a8c3-7a51-415a-b628-8634617eab10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.670479 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152a8c3-7a51-415a-b628-8634617eab10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.672069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152a8c3-7a51-415a-b628-8634617eab10 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.682611 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1431/2837] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:38:54.686864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f61a66-0f42-4184-9fb4-4a22ee572b5d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.697491 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f61a66-0f42-4184-9fb4-4a22ee572b5d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.702073 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3b7ad-9537-4c2c-9fc2-27c5b0484d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.702717 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b7ad-9537-4c2c-9fc2-27c5b0484d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.702996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b7ad-9537-4c2c-9fc2-27c5b0484d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.703298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b7ad-9537-4c2c-9fc2-27c5b0484d01 admin admin] PATH_INFO: `/v3/projects/41096bb76c3b47be9568d38a9f52766a/users/a48d0f7b67804548a9d047a12a806a84/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.705232 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3b7ad-9537-4c2c-9fc2-27c5b0484d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41096bb76c3b47be9568d38a9f52766a, user_id=a48d0f7b67804548a9d047a12a806a84, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.707880 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3b7ad-9537-4c2c-9fc2-27c5b0484d01 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.718206 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1407/2838] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:38:54 2022] PUT /identity/v3/projects/41096bb76c3b47be9568d38a9f52766a/users/a48d0f7b67804548a9d047a12a806a84/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:38:54.728994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf0d62-05ab-41d1-99bc-9a03974db8ff None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.729279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf0d62-05ab-41d1-99bc-9a03974db8ff None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.729549 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf0d62-05ab-41d1-99bc-9a03974db8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.757120 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-eabf0d62-05ab-41d1-99bc-9a03974db8ff None None] MFA Rules not processed for user `a48d0f7b67804548a9d047a12a806a84`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:38:54.758326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-eabf0d62-05ab-41d1-99bc-9a03974db8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:54.826280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1432/2839] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:38:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:38:54.837465 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3b7ad-9537-4c2c-9fc2-27c5b0484d01 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:54.839159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-f9e3b7ad-9537-4c2c-9fc2-27c5b0484d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:38:54.897869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3b7ad-9537-4c2c-9fc2-27c5b0484d01 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:54.901447 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520bb13-278e-4421-b7d7-60ec2508dd5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:54.902046 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520bb13-278e-4421-b7d7-60ec2508dd5c service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:54.902322 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520bb13-278e-4421-b7d7-60ec2508dd5c service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:54.902590 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520bb13-278e-4421-b7d7-60ec2508dd5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:54.958067 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520bb13-278e-4421-b7d7-60ec2508dd5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:54.960135 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520bb13-278e-4421-b7d7-60ec2508dd5c service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:54.969344 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1408/2840] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:38:56.467473 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf0d62-05ab-41d1-99bc-9a03974db8ff None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:56.477016 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf0d62-05ab-41d1-99bc-9a03974db8ff None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:56.480490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f312fc-06e9-4eca-b277-3bf531700319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:56.481100 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f312fc-06e9-4eca-b277-3bf531700319 service cinder] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:56.481370 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f312fc-06e9-4eca-b277-3bf531700319 service cinder] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:56.481637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f312fc-06e9-4eca-b277-3bf531700319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:56.537155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f312fc-06e9-4eca-b277-3bf531700319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:56.538712 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f312fc-06e9-4eca-b277-3bf531700319 service cinder] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:56.547493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1433/2841] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:38:57.074261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520bb13-278e-4421-b7d7-60ec2508dd5c service cinder] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:38:57.083349 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520bb13-278e-4421-b7d7-60ec2508dd5c service cinder] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:38:57.087024 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b743f3f-cbdc-4b6c-b943-645f9435df67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:38:57.087619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b743f3f-cbdc-4b6c-b943-645f9435df67 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:38:57.087897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b743f3f-cbdc-4b6c-b943-645f9435df67 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:38:57.088170 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b743f3f-cbdc-4b6c-b943-645f9435df67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:38:57.097193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b743f3f-cbdc-4b6c-b943-645f9435df67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:38:57.098619 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b743f3f-cbdc-4b6c-b943-645f9435df67 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:38:57.108411 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1409/2842] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:38:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:39:00.961668 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f312fc-06e9-4eca-b277-3bf531700319 service cinder] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:01.023038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f312fc-06e9-4eca-b277-3bf531700319 service cinder] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:01.027442 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:01.028050 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:01.028324 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:01.028619 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] PATH_INFO: `/v3/users/fca8cb665f5841609a5e271cbd100003` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:01.029701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca8cb665f5841609a5e271cbd100003)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:01.031614 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:01.036156 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:01.078007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca8cb665f5841609a5e271cbd100003'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.078651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca8cb665f5841609a5e271cbd100003'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.086753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca8cb665f5841609a5e271cbd100003'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.093797 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca8cb665f5841609a5e271cbd100003'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.094415 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca8cb665f5841609a5e271cbd100003'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.112515 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1434/2843] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:39:00 2022] DELETE /identity/v3/users/fca8cb665f5841609a5e271cbd100003 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:01.117157 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc901e-cf06-43e9-bc8e-04d16adddfa5 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:01.117630 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc901e-cf06-43e9-bc8e-04d16adddfa5 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:01.117763 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc901e-cf06-43e9-bc8e-04d16adddfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:01.136433 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-47dc901e-cf06-43e9-bc8e-04d16adddfa5 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:01.137668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-47dc901e-cf06-43e9-bc8e-04d16adddfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:01.195956 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1410/2844] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:01.205937 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:01.207332 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:01.270514 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809d7e2-1d26-42f2-892f-bffabb401e08 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:01.275093 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168ea61-8219-4387-9d3e-bb47642e6d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:01.275945 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168ea61-8219-4387-9d3e-bb47642e6d4d service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:01.276428 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168ea61-8219-4387-9d3e-bb47642e6d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:01.276958 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168ea61-8219-4387-9d3e-bb47642e6d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:01.287535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168ea61-8219-4387-9d3e-bb47642e6d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:01.289146 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168ea61-8219-4387-9d3e-bb47642e6d4d service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:01.298505 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1435/2845] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:39:01.564048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc901e-cf06-43e9-bc8e-04d16adddfa5 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:01.565592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-47dc901e-cf06-43e9-bc8e-04d16adddfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:01.583801 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc901e-cf06-43e9-bc8e-04d16adddfa5 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:01.588929 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:01.589599 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:01.589881 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:01.590240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] PATH_INFO: `/v3/projects/c83af4d1c60a4d00a5a8ed1b28dd3294` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:01.591292 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83af4d1c60a4d00a5a8ed1b28dd3294)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:01.593398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:01.613165 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:01.648507 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83af4d1c60a4d00a5a8ed1b28dd3294', 'request_id': 'req-acd67975-2665-42c8-b9ed-8071c574762b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.654210 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83af4d1c60a4d00a5a8ed1b28dd3294', 'request_id': 'req-acd67975-2665-42c8-b9ed-8071c574762b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.660472 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83af4d1c60a4d00a5a8ed1b28dd3294', 'request_id': 'req-acd67975-2665-42c8-b9ed-8071c574762b'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.674392 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] The token cache is being invalidate because project c83af4d1c60a4d00a5a8ed1b28dd3294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:39:01.674769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.675401 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.676935 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1411/2846] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:39:01 2022] DELETE /identity/v3/projects/c83af4d1c60a4d00a5a8ed1b28dd3294 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:01.681794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168ea61-8219-4387-9d3e-bb47642e6d4d service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:01.693007 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168ea61-8219-4387-9d3e-bb47642e6d4d service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:01.750096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:01.750721 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:01.750994 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:01.751263 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] PATH_INFO: `/v3/users/ac1dd6b56cfe4814a1583cd2ad43c071` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:01.752377 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1dd6b56cfe4814a1583cd2ad43c071)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:01.754333 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:01.782954 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:01.802090 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1dd6b56cfe4814a1583cd2ad43c071'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.802744 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1dd6b56cfe4814a1583cd2ad43c071'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.809976 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1dd6b56cfe4814a1583cd2ad43c071'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.816774 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1dd6b56cfe4814a1583cd2ad43c071'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.817397 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1dd6b56cfe4814a1583cd2ad43c071'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:01.834687 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1436/2847] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:39:01 2022] DELETE /identity/v3/users/ac1dd6b56cfe4814a1583cd2ad43c071 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:02.104352 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:02.105925 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:02.120091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd67975-2665-42c8-b9ed-8071c574762b admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:02.181347 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:02.181990 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:02.182267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:02.182535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] PATH_INFO: `/v3/projects/e996df60c24c4f7cb166d3d17ccf41b3` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:02.183729 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e996df60c24c4f7cb166d3d17ccf41b3)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:02.185724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:02.243748 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e996df60c24c4f7cb166d3d17ccf41b3', 'request_id': 'req-27b59754-e25d-4fc7-b2a9-598098e7f88c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.254607 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e996df60c24c4f7cb166d3d17ccf41b3', 'request_id': 'req-27b59754-e25d-4fc7-b2a9-598098e7f88c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.262444 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e996df60c24c4f7cb166d3d17ccf41b3', 'request_id': 'req-27b59754-e25d-4fc7-b2a9-598098e7f88c'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.278431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] The token cache is being invalidate because project e996df60c24c4f7cb166d3d17ccf41b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:39:02.278730 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.279410 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.280977 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1412/2848] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:39:02 2022] DELETE /identity/v3/projects/e996df60c24c4f7cb166d3d17ccf41b3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:02.285018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:02.286446 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:02.299724 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13632-17f1-454b-a7d0-ccdd465a6c03 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:02.360000 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:02.360584 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:02.360866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:02.361139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] PATH_INFO: `/v3/users/a48d0f7b67804548a9d047a12a806a84` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:02.362314 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48d0f7b67804548a9d047a12a806a84)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:02.364279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:02.414727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48d0f7b67804548a9d047a12a806a84'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.415387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48d0f7b67804548a9d047a12a806a84'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.423291 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48d0f7b67804548a9d047a12a806a84'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.431192 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48d0f7b67804548a9d047a12a806a84'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.431914 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-dae17d47-3f6b-4f25-bab3-0269377107ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48d0f7b67804548a9d047a12a806a84'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.451108 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1437/2849] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:39:02 2022] DELETE /identity/v3/users/a48d0f7b67804548a9d047a12a806a84 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:02.732066 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:02.733608 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:02.765060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b59754-e25d-4fc7-b2a9-598098e7f88c admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:02.834169 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:02.877907 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:02.878569 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:02.878854 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:02.879131 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] PATH_INFO: `/v3/projects/41096bb76c3b47be9568d38a9f52766a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:02.880290 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41096bb76c3b47be9568d38a9f52766a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:02.883678 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:02.935891 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41096bb76c3b47be9568d38a9f52766a', 'request_id': 'req-da459f64-8ca3-4c34-bed2-148b65515d40'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.941721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41096bb76c3b47be9568d38a9f52766a', 'request_id': 'req-da459f64-8ca3-4c34-bed2-148b65515d40'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.948967 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41096bb76c3b47be9568d38a9f52766a', 'request_id': 'req-da459f64-8ca3-4c34-bed2-148b65515d40'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.962805 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] The token cache is being invalidate because project 41096bb76c3b47be9568d38a9f52766a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:39:02.963164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.963920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-da459f64-8ca3-4c34-bed2-148b65515d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:02.965596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1413/2850] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:39:02 2022] DELETE /identity/v3/projects/41096bb76c3b47be9568d38a9f52766a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:02.970752 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885e05a-b19c-40d6-8c51-29a72b8f917a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:02.971040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885e05a-b19c-40d6-8c51-29a72b8f917a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:02.971316 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885e05a-b19c-40d6-8c51-29a72b8f917a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:02.992962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-2885e05a-b19c-40d6-8c51-29a72b8f917a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:02.994161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2885e05a-b19c-40d6-8c51-29a72b8f917a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:03.012579 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:03.058182 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1438/2851] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:03.067069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd89606-40a4-4689-a86c-1cd9c35c79e2 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.067360 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd89606-40a4-4689-a86c-1cd9c35c79e2 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.067635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd89606-40a4-4689-a86c-1cd9c35c79e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.088281 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-9dd89606-40a4-4689-a86c-1cd9c35c79e2 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:03.089514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-9dd89606-40a4-4689-a86c-1cd9c35c79e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:03.097270 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1414/2852] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:03.101962 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885e05a-b19c-40d6-8c51-29a72b8f917a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:03.111438 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885e05a-b19c-40d6-8c51-29a72b8f917a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:03.115850 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348a271-7d33-4e55-8cf0-7afab591de3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:03.117172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348a271-7d33-4e55-8cf0-7afab591de3d admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.117440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348a271-7d33-4e55-8cf0-7afab591de3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.117704 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348a271-7d33-4e55-8cf0-7afab591de3d admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.118340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348a271-7d33-4e55-8cf0-7afab591de3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:39:03.118631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348a271-7d33-4e55-8cf0-7afab591de3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:03.120201 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348a271-7d33-4e55-8cf0-7afab591de3d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:03.131449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1439/2853] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 00:39:03.135700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd89606-40a4-4689-a86c-1cd9c35c79e2 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:03.144849 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd89606-40a4-4689-a86c-1cd9c35c79e2 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:03.149513 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cf51f-fe6f-4e86-b8bb-c8a7b4c1bb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:03.150095 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cf51f-fe6f-4e86-b8bb-c8a7b4c1bb60 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.150432 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cf51f-fe6f-4e86-b8bb-c8a7b4c1bb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.150635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cf51f-fe6f-4e86-b8bb-c8a7b4c1bb60 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.151263 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cf51f-fe6f-4e86-b8bb-c8a7b4c1bb60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:03.153205 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cf51f-fe6f-4e86-b8bb-c8a7b4c1bb60 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:03.180553 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1415/2854] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:39:03 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:39:03.184750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348a271-7d33-4e55-8cf0-7afab591de3d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:03.194707 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348a271-7d33-4e55-8cf0-7afab591de3d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:03.250856 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239cfff-e2e2-4f7b-bc77-5e3e6b0d6651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:03.251576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239cfff-e2e2-4f7b-bc77-5e3e6b0d6651 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.251861 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239cfff-e2e2-4f7b-bc77-5e3e6b0d6651 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.252130 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239cfff-e2e2-4f7b-bc77-5e3e6b0d6651 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.252788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239cfff-e2e2-4f7b-bc77-5e3e6b0d6651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:03.254757 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239cfff-e2e2-4f7b-bc77-5e3e6b0d6651 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:03.276416 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1440/2855] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:39:03 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:39:03.281312 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cf51f-fe6f-4e86-b8bb-c8a7b4c1bb60 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:03.292952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cf51f-fe6f-4e86-b8bb-c8a7b4c1bb60 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:03.297718 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfd27d-9989-4582-95ce-6cd6a98f831f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:03.298336 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfd27d-9989-4582-95ce-6cd6a98f831f admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.298596 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfd27d-9989-4582-95ce-6cd6a98f831f admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.299000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfd27d-9989-4582-95ce-6cd6a98f831f admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.299675 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfd27d-9989-4582-95ce-6cd6a98f831f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:03.301659 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfd27d-9989-4582-95ce-6cd6a98f831f admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:03.315647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1416/2856] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:39:03.320049 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239cfff-e2e2-4f7b-bc77-5e3e6b0d6651 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:03.331535 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239cfff-e2e2-4f7b-bc77-5e3e6b0d6651 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:03.338325 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d498b-6713-4248-b9eb-874c7138f48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:03.339014 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d498b-6713-4248-b9eb-874c7138f48b admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.339361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d498b-6713-4248-b9eb-874c7138f48b admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.339702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d498b-6713-4248-b9eb-874c7138f48b admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.340413 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d498b-6713-4248-b9eb-874c7138f48b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:03.342113 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d498b-6713-4248-b9eb-874c7138f48b admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:03.354106 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1441/2857] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:39:03.358310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfd27d-9989-4582-95ce-6cd6a98f831f admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:03.370117 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfd27d-9989-4582-95ce-6cd6a98f831f admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:03.375752 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa258346-8600-40e4-bc55-c659b0dbfbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:03.376543 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa258346-8600-40e4-bc55-c659b0dbfbaf admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.376732 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa258346-8600-40e4-bc55-c659b0dbfbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.377020 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa258346-8600-40e4-bc55-c659b0dbfbaf admin admin] PATH_INFO: `/v3/projects/859543cced6c4e229b1525039e478cf8/users/00a0d21417dd47e0b56fe3b38ce03373/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.395153 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa258346-8600-40e4-bc55-c659b0dbfbaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=859543cced6c4e229b1525039e478cf8, user_id=00a0d21417dd47e0b56fe3b38ce03373, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:03.399099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa258346-8600-40e4-bc55-c659b0dbfbaf admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:03.417735 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1417/2858] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:39:03 2022] PUT /identity/v3/projects/859543cced6c4e229b1525039e478cf8/users/00a0d21417dd47e0b56fe3b38ce03373/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:03.428685 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f701915-e00e-4a80-a355-94316e99a5ed None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.429699 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f701915-e00e-4a80-a355-94316e99a5ed None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.429992 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f701915-e00e-4a80-a355-94316e99a5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.462973 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-4f701915-e00e-4a80-a355-94316e99a5ed None None] MFA Rules not processed for user `00a0d21417dd47e0b56fe3b38ce03373`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:03.464218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4f701915-e00e-4a80-a355-94316e99a5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:03.529895 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1442/2859] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:03.541897 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa258346-8600-40e4-bc55-c659b0dbfbaf admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:03.543616 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-fa258346-8600-40e4-bc55-c659b0dbfbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:03.601428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa258346-8600-40e4-bc55-c659b0dbfbaf admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:03.605558 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff3f13-1f0a-4960-aa42-a69cfbbc2de4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5590d8961d47aa846117a236730d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:03.606218 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff3f13-1f0a-4960-aa42-a69cfbbc2de4 service cinder] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.606501 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff3f13-1f0a-4960-aa42-a69cfbbc2de4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.606723 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff3f13-1f0a-4960-aa42-a69cfbbc2de4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.615627 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff3f13-1f0a-4960-aa42-a69cfbbc2de4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:03.617810 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff3f13-1f0a-4960-aa42-a69cfbbc2de4 service cinder] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:03.626843 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1418/2860] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:39:03.652647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f701915-e00e-4a80-a355-94316e99a5ed None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:03.674963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:03.720481 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f701915-e00e-4a80-a355-94316e99a5ed None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:03.725753 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:03.726406 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.726684 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.726960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] PATH_INFO: `/v3/users/00a0d21417dd47e0b56fe3b38ce03373` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.728105 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00a0d21417dd47e0b56fe3b38ce03373)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:03.730073 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:03.775995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00a0d21417dd47e0b56fe3b38ce03373'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:03.776788 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00a0d21417dd47e0b56fe3b38ce03373'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:03.784344 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00a0d21417dd47e0b56fe3b38ce03373'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:03.791963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00a0d21417dd47e0b56fe3b38ce03373'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:03.792635 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00a0d21417dd47e0b56fe3b38ce03373'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:03.810782 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1443/2861] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:39:03 2022] DELETE /identity/v3/users/00a0d21417dd47e0b56fe3b38ce03373 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:03.815298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de985e5-aa29-4100-b145-1a2d5e0ff639 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.815586 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de985e5-aa29-4100-b145-1a2d5e0ff639 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.815864 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de985e5-aa29-4100-b145-1a2d5e0ff639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.823605 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:03.837317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-2de985e5-aa29-4100-b145-1a2d5e0ff639 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:03.838525 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2de985e5-aa29-4100-b145-1a2d5e0ff639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:03.897372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1419/2862] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:03.907029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:03.908440 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:03.968807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d775e-206d-4e2f-929d-4277e2a33e4f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:03.972394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f73bf-9a15-41c0-a136-daf2e4393e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:03.973029 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f73bf-9a15-41c0-a136-daf2e4393e2f service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:03.973307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f73bf-9a15-41c0-a136-daf2e4393e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:03.973578 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f73bf-9a15-41c0-a136-daf2e4393e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:03.984964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f73bf-9a15-41c0-a136-daf2e4393e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:03.986425 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f73bf-9a15-41c0-a136-daf2e4393e2f service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:03.999249 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1444/2863] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:39:04.304436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de985e5-aa29-4100-b145-1a2d5e0ff639 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:04.305923 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-2de985e5-aa29-4100-b145-1a2d5e0ff639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:04.319113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de985e5-aa29-4100-b145-1a2d5e0ff639 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:04.324943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:04.325585 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.326383 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.326661 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] PATH_INFO: `/v3/projects/859543cced6c4e229b1525039e478cf8` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.327982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=859543cced6c4e229b1525039e478cf8)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:04.329944 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:04.378947 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '859543cced6c4e229b1525039e478cf8', 'request_id': 'req-d7c01f89-2d9e-47ae-a840-7b649eb78dca'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:04.385620 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '859543cced6c4e229b1525039e478cf8', 'request_id': 'req-d7c01f89-2d9e-47ae-a840-7b649eb78dca'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:04.392111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '859543cced6c4e229b1525039e478cf8', 'request_id': 'req-d7c01f89-2d9e-47ae-a840-7b649eb78dca'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:04.405109 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] The token cache is being invalidate because project 859543cced6c4e229b1525039e478cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:39:04.405359 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:04.405984 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-d7c01f89-2d9e-47ae-a840-7b649eb78dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:04.407461 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1420/2864] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:39:04 2022] DELETE /identity/v3/projects/859543cced6c4e229b1525039e478cf8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:04.412647 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1a639-0a97-4199-bbe6-3182a4587b0a None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.413651 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1a639-0a97-4199-bbe6-3182a4587b0a None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.413970 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1a639-0a97-4199-bbe6-3182a4587b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.433632 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-d9c1a639-0a97-4199-bbe6-3182a4587b0a None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:04.434827 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d9c1a639-0a97-4199-bbe6-3182a4587b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:04.473560 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:04.498160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1445/2865] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:04.506254 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be1bb8-0852-4f42-afbf-a06b458ff33e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.506632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be1bb8-0852-4f42-afbf-a06b458ff33e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.506908 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be1bb8-0852-4f42-afbf-a06b458ff33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.527100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-b0be1bb8-0852-4f42-afbf-a06b458ff33e None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:04.528284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b0be1bb8-0852-4f42-afbf-a06b458ff33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:04.536134 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1421/2866] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:04.541020 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1a639-0a97-4199-bbe6-3182a4587b0a None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:04.550520 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1a639-0a97-4199-bbe6-3182a4587b0a None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:04.555087 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cef8a-c920-4fc1-9a04-0de5db384086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:04.555680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cef8a-c920-4fc1-9a04-0de5db384086 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.555957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cef8a-c920-4fc1-9a04-0de5db384086 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.556227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007cef8a-c920-4fc1-9a04-0de5db384086 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.556924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007cef8a-c920-4fc1-9a04-0de5db384086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:39:04.557221 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007cef8a-c920-4fc1-9a04-0de5db384086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:04.558812 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007cef8a-c920-4fc1-9a04-0de5db384086 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:04.574322 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1446/2867] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:39:04.575556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be1bb8-0852-4f42-afbf-a06b458ff33e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:04.585537 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be1bb8-0852-4f42-afbf-a06b458ff33e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:04.590307 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f45f95-70f4-469b-9569-ee8bcba16122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:04.590905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f45f95-70f4-469b-9569-ee8bcba16122 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.591184 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f45f95-70f4-469b-9569-ee8bcba16122 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.591458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f45f95-70f4-469b-9569-ee8bcba16122 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.592127 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f45f95-70f4-469b-9569-ee8bcba16122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:04.594069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f45f95-70f4-469b-9569-ee8bcba16122 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:04.620126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1422/2868] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:39:04 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:39:04.624486 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cef8a-c920-4fc1-9a04-0de5db384086 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:04.635293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007cef8a-c920-4fc1-9a04-0de5db384086 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:04.691890 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7c673-c4aa-42c8-bdaf-4516af97167a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:04.692419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7c673-c4aa-42c8-bdaf-4516af97167a admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.692729 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7c673-c4aa-42c8-bdaf-4516af97167a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.693005 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca7c673-c4aa-42c8-bdaf-4516af97167a admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.693641 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7c673-c4aa-42c8-bdaf-4516af97167a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:04.696457 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca7c673-c4aa-42c8-bdaf-4516af97167a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:04.717308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1447/2869] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:39:04 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:39:04.721350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f45f95-70f4-469b-9569-ee8bcba16122 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:04.731562 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f45f95-70f4-469b-9569-ee8bcba16122 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:04.736177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46a498-821c-4623-b113-a91fa8d61def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:04.736809 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46a498-821c-4623-b113-a91fa8d61def admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.737093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46a498-821c-4623-b113-a91fa8d61def admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.737368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46a498-821c-4623-b113-a91fa8d61def admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.738009 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46a498-821c-4623-b113-a91fa8d61def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:04.739615 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46a498-821c-4623-b113-a91fa8d61def admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:04.750001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1423/2870] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:39:04.753776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7c673-c4aa-42c8-bdaf-4516af97167a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:04.764290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca7c673-c4aa-42c8-bdaf-4516af97167a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:04.769018 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915019f9-759b-4af0-99a1-8f788b87c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:04.769602 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915019f9-759b-4af0-99a1-8f788b87c6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.769875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915019f9-759b-4af0-99a1-8f788b87c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.770158 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915019f9-759b-4af0-99a1-8f788b87c6d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.770823 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915019f9-759b-4af0-99a1-8f788b87c6d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:04.772935 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915019f9-759b-4af0-99a1-8f788b87c6d1 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:04.782777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1448/2871] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:39:04.786840 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46a498-821c-4623-b113-a91fa8d61def admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:04.796759 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46a498-821c-4623-b113-a91fa8d61def admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:04.801317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a30b8-d82f-4f20-b9e4-de749ccc05d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:04.801954 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a30b8-d82f-4f20-b9e4-de749ccc05d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.802229 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a30b8-d82f-4f20-b9e4-de749ccc05d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.802496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a30b8-d82f-4f20-b9e4-de749ccc05d2 admin admin] PATH_INFO: `/v3/projects/a22a4b4aab87497eb1f8b1bce8cd17d5/users/d7396c3776b744328babae60d6cfb6fd/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.817256 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a30b8-d82f-4f20-b9e4-de749ccc05d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22a4b4aab87497eb1f8b1bce8cd17d5, user_id=d7396c3776b744328babae60d6cfb6fd, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:04.820737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a30b8-d82f-4f20-b9e4-de749ccc05d2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:04.833331 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1424/2872] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:39:04 2022] PUT /identity/v3/projects/a22a4b4aab87497eb1f8b1bce8cd17d5/users/d7396c3776b744328babae60d6cfb6fd/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:04.838498 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87aec2-de30-431d-bd7f-dfcd351dd1fe None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.838807 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87aec2-de30-431d-bd7f-dfcd351dd1fe None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.839085 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87aec2-de30-431d-bd7f-dfcd351dd1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:04.859160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-4a87aec2-de30-431d-bd7f-dfcd351dd1fe None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:04.860369 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-4a87aec2-de30-431d-bd7f-dfcd351dd1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:04.922407 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1449/2873] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:04.932589 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a30b8-d82f-4f20-b9e4-de749ccc05d2 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:04.934295 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ba1a30b8-d82f-4f20-b9e4-de749ccc05d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:04.961802 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:04.994556 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a30b8-d82f-4f20-b9e4-de749ccc05d2 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:04.998284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6b51f-786b-4b79-a10a-f3545f6c68b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:04.998887 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6b51f-786b-4b79-a10a-f3545f6c68b4 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:04.999164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6b51f-786b-4b79-a10a-f3545f6c68b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:04.999436 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6b51f-786b-4b79-a10a-f3545f6c68b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:05.009148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6b51f-786b-4b79-a10a-f3545f6c68b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:05.010624 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6b51f-786b-4b79-a10a-f3545f6c68b4 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:05.019981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1425/2874] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:39:08.809181 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed178c-7423-48aa-b585-91de082733a0 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:08.809692 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed178c-7423-48aa-b585-91de082733a0 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:08.809801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed178c-7423-48aa-b585-91de082733a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:08.860777 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-85ed178c-7423-48aa-b585-91de082733a0 None None] MFA Rules not processed for user `d7396c3776b744328babae60d6cfb6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:08.862399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-85ed178c-7423-48aa-b585-91de082733a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:08.945252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:08.982882 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1450/2875] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 00:39:08.987368 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6b51f-786b-4b79-a10a-f3545f6c68b4 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:09.019079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6b51f-786b-4b79-a10a-f3545f6c68b4 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:09.028249 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959212b-80a4-4fa4-b2e9-550db672192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:09.028998 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959212b-80a4-4fa4-b2e9-550db672192d admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:09.029340 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959212b-80a4-4fa4-b2e9-550db672192d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:09.029693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959212b-80a4-4fa4-b2e9-550db672192d admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:09.030389 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959212b-80a4-4fa4-b2e9-550db672192d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:09.032382 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959212b-80a4-4fa4-b2e9-550db672192d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:09.064635 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1426/2876] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:39:08 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:39:09.069543 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed178c-7423-48aa-b585-91de082733a0 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:09.083622 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed178c-7423-48aa-b585-91de082733a0 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:09.151175 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a60ff2-e3d1-40b2-9c67-9f3a247e9c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:09.151852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a60ff2-e3d1-40b2-9c67-9f3a247e9c4f admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:09.152234 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a60ff2-e3d1-40b2-9c67-9f3a247e9c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:09.152702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a60ff2-e3d1-40b2-9c67-9f3a247e9c4f admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:09.153454 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a60ff2-e3d1-40b2-9c67-9f3a247e9c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:09.155717 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a60ff2-e3d1-40b2-9c67-9f3a247e9c4f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:09.180387 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1451/2877] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:39:09 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:39:09.184622 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959212b-80a4-4fa4-b2e9-550db672192d admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:09.195623 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959212b-80a4-4fa4-b2e9-550db672192d admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:09.200742 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb9738-a5d0-4440-88a1-02b24d1928d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:09.201723 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb9738-a5d0-4440-88a1-02b24d1928d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:09.202163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb9738-a5d0-4440-88a1-02b24d1928d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:09.202672 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb9738-a5d0-4440-88a1-02b24d1928d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:09.203951 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb9738-a5d0-4440-88a1-02b24d1928d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:09.205991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb9738-a5d0-4440-88a1-02b24d1928d9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:09.220042 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1427/2878] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:39:09.224307 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a60ff2-e3d1-40b2-9c67-9f3a247e9c4f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:09.238826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a60ff2-e3d1-40b2-9c67-9f3a247e9c4f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:09.249123 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb9b6-f9f8-482c-94f1-689c3cc6420f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:09.250750 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb9b6-f9f8-482c-94f1-689c3cc6420f admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:09.251229 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb9b6-f9f8-482c-94f1-689c3cc6420f admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:09.251605 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bb9b6-f9f8-482c-94f1-689c3cc6420f admin admin] PATH_INFO: `/v3/projects/af42a4ea101646048c6f453a555b5093/users/c5a0070ed2ef453dae0e93b8ddaf0870/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:09.271380 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb9b6-f9f8-482c-94f1-689c3cc6420f admin admin] RBAC: Authorizing `identity:create_grant(project_id=af42a4ea101646048c6f453a555b5093, user_id=c5a0070ed2ef453dae0e93b8ddaf0870, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:09.274979 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bb9b6-f9f8-482c-94f1-689c3cc6420f admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:09.290792 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1452/2879] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:39:09 2022] PUT /identity/v3/projects/af42a4ea101646048c6f453a555b5093/users/c5a0070ed2ef453dae0e93b8ddaf0870/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:09.294894 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb9738-a5d0-4440-88a1-02b24d1928d9 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:09.305164 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb9738-a5d0-4440-88a1-02b24d1928d9 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:09.375381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c13076-7c1c-4d3a-b65a-8e8a553b7fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:09.375994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c13076-7c1c-4d3a-b65a-8e8a553b7fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:09.376267 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c13076-7c1c-4d3a-b65a-8e8a553b7fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:09.376576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c13076-7c1c-4d3a-b65a-8e8a553b7fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:09.377220 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c13076-7c1c-4d3a-b65a-8e8a553b7fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:09.379001 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c13076-7c1c-4d3a-b65a-8e8a553b7fc6 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:09.390502 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1428/2880] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:39:09.394761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb9b6-f9f8-482c-94f1-689c3cc6420f admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:09.406725 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bb9b6-f9f8-482c-94f1-689c3cc6420f admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:09.411345 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5d7d0-f3ed-48cd-9fbe-b0463e5cd45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:09.411999 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5d7d0-f3ed-48cd-9fbe-b0463e5cd45e admin admin] REQUEST_METHOD: `PUT` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:09.412297 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5d7d0-f3ed-48cd-9fbe-b0463e5cd45e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:09.413562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5d7d0-f3ed-48cd-9fbe-b0463e5cd45e admin admin] PATH_INFO: `/v3/projects/af42a4ea101646048c6f453a555b5093/users/c5a0070ed2ef453dae0e93b8ddaf0870/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:09.415639 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5d7d0-f3ed-48cd-9fbe-b0463e5cd45e admin admin] RBAC: Authorizing `identity:create_grant(project_id=af42a4ea101646048c6f453a555b5093, user_id=c5a0070ed2ef453dae0e93b8ddaf0870, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:09.418399 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5d7d0-f3ed-48cd-9fbe-b0463e5cd45e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:09.428984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1453/2881] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:39:09 2022] PUT /identity/v3/projects/af42a4ea101646048c6f453a555b5093/users/c5a0070ed2ef453dae0e93b8ddaf0870/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:14.052792 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146ce7-e908-4262-aab2-8068de0ef287 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:14.053855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146ce7-e908-4262-aab2-8068de0ef287 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:14.054150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59146ce7-e908-4262-aab2-8068de0ef287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:14.092060 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-59146ce7-e908-4262-aab2-8068de0ef287 None None] MFA Rules not processed for user `c5a0070ed2ef453dae0e93b8ddaf0870`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:14.093412 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-59146ce7-e908-4262-aab2-8068de0ef287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:14.186526 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1429/2882] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:14.196489 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5d7d0-f3ed-48cd-9fbe-b0463e5cd45e admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:14.199555 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-30d5d7d0-f3ed-48cd-9fbe-b0463e5cd45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:14.270009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5d7d0-f3ed-48cd-9fbe-b0463e5cd45e admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:14.273944 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bccf2-0ec3-4991-b20f-a71842ebff64 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52470b620abd4cd29ab7e4752f285bb3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:14.274557 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bccf2-0ec3-4991-b20f-a71842ebff64 service glance] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:14.275256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bccf2-0ec3-4991-b20f-a71842ebff64 service glance] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:14.275256 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bccf2-0ec3-4991-b20f-a71842ebff64 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:14.334082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:14.351154 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bccf2-0ec3-4991-b20f-a71842ebff64 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:14.361310 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bccf2-0ec3-4991-b20f-a71842ebff64 service glance] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:14.374956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1454/2883] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:39:14.504699 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471f83e-e4f2-4da8-a7d2-a9ea47d752bf None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:14.505088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471f83e-e4f2-4da8-a7d2-a9ea47d752bf None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:14.505431 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471f83e-e4f2-4da8-a7d2-a9ea47d752bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:14.525512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:14.539091 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-d471f83e-e4f2-4da8-a7d2-a9ea47d752bf None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:14.540317 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-d471f83e-e4f2-4da8-a7d2-a9ea47d752bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:14.608430 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1430/2884] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:39:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:14.654056 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bccf2-0ec3-4991-b20f-a71842ebff64 service glance] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:14.655467 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2a2bccf2-0ec3-4991-b20f-a71842ebff64 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:14.726023 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bccf2-0ec3-4991-b20f-a71842ebff64 service glance] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:14.730984 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7bd66-8511-4660-ae05-bd8a4cae0a39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:14.731588 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7bd66-8511-4660-ae05-bd8a4cae0a39 service swift] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:14.731866 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7bd66-8511-4660-ae05-bd8a4cae0a39 service swift] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:14.732139 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7bd66-8511-4660-ae05-bd8a4cae0a39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:14.740044 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7bd66-8511-4660-ae05-bd8a4cae0a39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:14.741939 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7bd66-8511-4660-ae05-bd8a4cae0a39 service swift] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:14.749839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1455/2885] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:39:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:39:24.437517 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd6ed5-ec78-463b-8e47-b23dd9de3843 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:24.438014 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd6ed5-ec78-463b-8e47-b23dd9de3843 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:24.438450 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd6ed5-ec78-463b-8e47-b23dd9de3843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:24.459379 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-80fd6ed5-ec78-463b-8e47-b23dd9de3843 None None] MFA Rules not processed for user `d7396c3776b744328babae60d6cfb6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:24.460621 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-80fd6ed5-ec78-463b-8e47-b23dd9de3843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:24.467536 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1431/2886] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:24.746758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=39 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:27.665794 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7bd66-8511-4660-ae05-bd8a4cae0a39 service swift] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:27.668258 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f3d7bd66-8511-4660-ae05-bd8a4cae0a39 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:27.744122 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7bd66-8511-4660-ae05-bd8a4cae0a39 service swift] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:27.749161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:27.749630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:27.749916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:27.750187 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] PATH_INFO: `/v3/users/7d042aa1a703464eab4a440db9ce1374` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:27.751339 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d042aa1a703464eab4a440db9ce1374)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:27.753303 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:27.800808 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d042aa1a703464eab4a440db9ce1374'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:27.801451 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d042aa1a703464eab4a440db9ce1374'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:27.809311 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d042aa1a703464eab4a440db9ce1374'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:27.816200 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d042aa1a703464eab4a440db9ce1374'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:27.816853 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-408cc491-5557-423d-af96-a46cdac70817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d042aa1a703464eab4a440db9ce1374'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:27.835218 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1456/2887] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:39:27 2022] DELETE /identity/v3/users/7d042aa1a703464eab4a440db9ce1374 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:28.006323 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd6ed5-ec78-463b-8e47-b23dd9de3843 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:28.017839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd6ed5-ec78-463b-8e47-b23dd9de3843 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:28.078126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:28.078768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.079191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.079475 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] PATH_INFO: `/v3/projects/07af35e80c004621bd72ab7336930693` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.080691 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07af35e80c004621bd72ab7336930693)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:28.082774 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:28.133875 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07af35e80c004621bd72ab7336930693', 'request_id': 'req-0f78cc77-3b62-47f7-a296-f899b0fa83d9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:28.139324 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07af35e80c004621bd72ab7336930693', 'request_id': 'req-0f78cc77-3b62-47f7-a296-f899b0fa83d9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:28.147284 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07af35e80c004621bd72ab7336930693', 'request_id': 'req-0f78cc77-3b62-47f7-a296-f899b0fa83d9'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:28.160454 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] The token cache is being invalidate because project 07af35e80c004621bd72ab7336930693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:39:28.160775 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:28.161426 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-0f78cc77-3b62-47f7-a296-f899b0fa83d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:39:28.163139 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1432/2888] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:39:28 2022] DELETE /identity/v3/projects/07af35e80c004621bd72ab7336930693 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:28.167860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb29d0-6fc8-4004-94a0-da3ebf2e4f93 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.168138 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb29d0-6fc8-4004-94a0-da3ebf2e4f93 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.168417 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb29d0-6fc8-4004-94a0-da3ebf2e4f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.194580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-edeb29d0-6fc8-4004-94a0-da3ebf2e4f93 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:28.196203 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-edeb29d0-6fc8-4004-94a0-da3ebf2e4f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:28.278040 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1457/2889] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:28.287178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891cd8f-fb3a-4230-9ed6-0b52152db6e2 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.287550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891cd8f-fb3a-4230-9ed6-0b52152db6e2 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.287830 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891cd8f-fb3a-4230-9ed6-0b52152db6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.310983 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-5891cd8f-fb3a-4230-9ed6-0b52152db6e2 None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:28.312201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5891cd8f-fb3a-4230-9ed6-0b52152db6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:28.321939 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1433/2890] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:28.326563 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb29d0-6fc8-4004-94a0-da3ebf2e4f93 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:28.336562 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb29d0-6fc8-4004-94a0-da3ebf2e4f93 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:28.341927 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0cb8f-cc5f-4bff-b883-26ec8890fc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:28.343697 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0cb8f-cc5f-4bff-b883-26ec8890fc33 admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.343844 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0cb8f-cc5f-4bff-b883-26ec8890fc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.344069 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0cb8f-cc5f-4bff-b883-26ec8890fc33 admin admin] PATH_INFO: `/v3/domains` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.344769 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0cb8f-cc5f-4bff-b883-26ec8890fc33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93105) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 00:39:28.345141 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0cb8f-cc5f-4bff-b883-26ec8890fc33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:28.346849 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0cb8f-cc5f-4bff-b883-26ec8890fc33 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:28.359480 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1458/2891] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 1 00:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 00:39:28.363654 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891cd8f-fb3a-4230-9ed6-0b52152db6e2 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:28.375903 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891cd8f-fb3a-4230-9ed6-0b52152db6e2 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:28.381934 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb64fb-564a-421e-a6a9-eeefc7fe7792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:28.382539 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb64fb-564a-421e-a6a9-eeefc7fe7792 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.382815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb64fb-564a-421e-a6a9-eeefc7fe7792 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.383159 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb64fb-564a-421e-a6a9-eeefc7fe7792 admin admin] PATH_INFO: `/v3/projects` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.383770 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb64fb-564a-421e-a6a9-eeefc7fe7792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:28.385737 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb64fb-564a-421e-a6a9-eeefc7fe7792 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:28.408328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:28.417148 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1434/2892] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:39:28 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:39:28.421308 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0cb8f-cc5f-4bff-b883-26ec8890fc33 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:28.432216 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0cb8f-cc5f-4bff-b883-26ec8890fc33 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:28.436537 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:28.498670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c36b4-6250-4b0b-996e-e2c2a45555f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:28.499283 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c36b4-6250-4b0b-996e-e2c2a45555f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.499664 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c36b4-6250-4b0b-996e-e2c2a45555f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.500011 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c36b4-6250-4b0b-996e-e2c2a45555f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.500637 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c36b4-6250-4b0b-996e-e2c2a45555f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:28.502576 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c36b4-6250-4b0b-996e-e2c2a45555f8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:28.526128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1459/2893] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:39:28 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:39:28.530231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb64fb-564a-421e-a6a9-eeefc7fe7792 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:28.541113 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb64fb-564a-421e-a6a9-eeefc7fe7792 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:28.546132 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181feb5e-f2c5-4708-a267-c976601b3f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:28.546857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181feb5e-f2c5-4708-a267-c976601b3f9e admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.547145 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181feb5e-f2c5-4708-a267-c976601b3f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.547504 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181feb5e-f2c5-4708-a267-c976601b3f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.548163 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181feb5e-f2c5-4708-a267-c976601b3f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:28.549827 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181feb5e-f2c5-4708-a267-c976601b3f9e admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:28.551556 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fd064-3e9b-4526-bd3c-22b0bd65fa7c None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.551852 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fd064-3e9b-4526-bd3c-22b0bd65fa7c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.554364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fd064-3e9b-4526-bd3c-22b0bd65fa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.563850 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1435/2894] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 00:39:28.564936 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181feb5e-f2c5-4708-a267-c976601b3f9e admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:28.576099 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-396fd064-3e9b-4526-bd3c-22b0bd65fa7c None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:28.578010 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-396fd064-3e9b-4526-bd3c-22b0bd65fa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:28.578150 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181feb5e-f2c5-4708-a267-c976601b3f9e admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:28.584195 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ac415-0fe9-4270-a186-54a595668f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:28.584868 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ac415-0fe9-4270-a186-54a595668f12 admin admin] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.585153 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ac415-0fe9-4270-a186-54a595668f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.585425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ac415-0fe9-4270-a186-54a595668f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.586301 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ac415-0fe9-4270-a186-54a595668f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:28.587656 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ac415-0fe9-4270-a186-54a595668f12 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:28.599848 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1436/2895] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:39:28.603980 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ac415-0fe9-4270-a186-54a595668f12 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:28.615417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ac415-0fe9-4270-a186-54a595668f12 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:28.621512 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884b69f-7116-4ba2-8970-42cbc270aad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:28.622173 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884b69f-7116-4ba2-8970-42cbc270aad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.622452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884b69f-7116-4ba2-8970-42cbc270aad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.622769 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884b69f-7116-4ba2-8970-42cbc270aad7 admin admin] PATH_INFO: `/v3/projects/31e9903e4f16471d8057ee7364cf123b/users/048aba7c5b364b07abf7f6b8977d167a/roles/df2ecb74b9fe42089e380b16de5ad950` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.640337 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884b69f-7116-4ba2-8970-42cbc270aad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e9903e4f16471d8057ee7364cf123b, user_id=048aba7c5b364b07abf7f6b8977d167a, role_id=df2ecb74b9fe42089e380b16de5ad950)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:28.643373 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884b69f-7116-4ba2-8970-42cbc270aad7 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:28.648276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1460/2896] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 1 00:39:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:28.657587 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1437/2897] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:39:28 2022] PUT /identity/v3/projects/31e9903e4f16471d8057ee7364cf123b/users/048aba7c5b364b07abf7f6b8977d167a/roles/df2ecb74b9fe42089e380b16de5ad950 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:28.664242 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a058f-ab75-4ff2-ac4c-0ddccebce53f None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.664242 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a058f-ab75-4ff2-ac4c-0ddccebce53f None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.664242 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a058f-ab75-4ff2-ac4c-0ddccebce53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.667668 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884b69f-7116-4ba2-8970-42cbc270aad7 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:28.669479 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4884b69f-7116-4ba2-8970-42cbc270aad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:28.685758 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-3f9a058f-ab75-4ff2-ac4c-0ddccebce53f None None] MFA Rules not processed for user `6789433c4ccf40c89d963e435a40ba65`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:28.686953 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3f9a058f-ab75-4ff2-ac4c-0ddccebce53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:28.754693 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884b69f-7116-4ba2-8970-42cbc270aad7 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:28.764110 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1461/2898] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:28.764428 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608dd26-5c71-49cd-92d2-623f620996cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:28.765051 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608dd26-5c71-49cd-92d2-623f620996cb service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.765240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608dd26-5c71-49cd-92d2-623f620996cb service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.765511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608dd26-5c71-49cd-92d2-623f620996cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.770613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a058f-ab75-4ff2-ac4c-0ddccebce53f None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:28.772997 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-3f9a058f-ab75-4ff2-ac4c-0ddccebce53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:28.829240 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608dd26-5c71-49cd-92d2-623f620996cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:28.830836 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608dd26-5c71-49cd-92d2-623f620996cb service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:28.830960 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:28.844910 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1438/2899] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:39:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 00:39:28.847713 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a058f-ab75-4ff2-ac4c-0ddccebce53f None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:28.853851 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e299c-207d-46fd-937d-5a5846eac10c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:28.854553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e299c-207d-46fd-937d-5a5846eac10c service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:28.854839 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e299c-207d-46fd-937d-5a5846eac10c service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:28.855112 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e299c-207d-46fd-937d-5a5846eac10c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:28.866536 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e299c-207d-46fd-937d-5a5846eac10c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:28.874957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e299c-207d-46fd-937d-5a5846eac10c service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:28.887554 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1462/2900] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:39:36.084208 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22f804-2a29-4d19-bb62-65b36744b950 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:36.084824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22f804-2a29-4d19-bb62-65b36744b950 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:36.084824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22f804-2a29-4d19-bb62-65b36744b950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:36.132877 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-4f22f804-2a29-4d19-bb62-65b36744b950 None None] MFA Rules not processed for user `048aba7c5b364b07abf7f6b8977d167a`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:36.135079 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-4f22f804-2a29-4d19-bb62-65b36744b950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:36.198514 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:36.225682 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1439/2901] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:36.233845 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e299c-207d-46fd-937d-5a5846eac10c service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:36.246657 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e299c-207d-46fd-937d-5a5846eac10c service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:36.256165 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bf369-05c9-4763-917d-b1e68768730a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:36.256806 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bf369-05c9-4763-917d-b1e68768730a admin admin] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:36.257082 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bf369-05c9-4763-917d-b1e68768730a admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:36.257350 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bf369-05c9-4763-917d-b1e68768730a admin admin] PATH_INFO: `/v3/projects` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:36.257983 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160bf369-05c9-4763-917d-b1e68768730a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:36.259905 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160bf369-05c9-4763-917d-b1e68768730a admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:36.297096 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1463/2902] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 1 00:39:36 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 00:39:36.301296 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22f804-2a29-4d19-bb62-65b36744b950 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:36.313524 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22f804-2a29-4d19-bb62-65b36744b950 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:36.377285 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1f3ac-a43e-4640-bab2-c7ad24d44424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:36.378156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f3ac-a43e-4640-bab2-c7ad24d44424 admin admin] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:36.378279 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f3ac-a43e-4640-bab2-c7ad24d44424 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:36.378547 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1f3ac-a43e-4640-bab2-c7ad24d44424 admin admin] PATH_INFO: `/v3/users` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:36.379631 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1f3ac-a43e-4640-bab2-c7ad24d44424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:36.382286 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1f3ac-a43e-4640-bab2-c7ad24d44424 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:36.414155 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bf369-05c9-4763-917d-b1e68768730a admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:36.414834 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1440/2903] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 1 00:39:36 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 00:39:36.426547 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bf369-05c9-4763-917d-b1e68768730a admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:36.438030 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44509309-fb58-440f-a7dd-2116a1f6384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:36.438659 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509309-fb58-440f-a7dd-2116a1f6384d admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:36.438957 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509309-fb58-440f-a7dd-2116a1f6384d admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:36.439225 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509309-fb58-440f-a7dd-2116a1f6384d admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:36.440134 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509309-fb58-440f-a7dd-2116a1f6384d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:36.441949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509309-fb58-440f-a7dd-2116a1f6384d admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:36.454995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1464/2904] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:39:36.459626 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1f3ac-a43e-4640-bab2-c7ad24d44424 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:36.471182 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1f3ac-a43e-4640-bab2-c7ad24d44424 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:36.481435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b61d0b-69e2-41c9-bb6d-5f1d5451d007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:36.482231 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b61d0b-69e2-41c9-bb6d-5f1d5451d007 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:36.482720 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b61d0b-69e2-41c9-bb6d-5f1d5451d007 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:36.483293 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b61d0b-69e2-41c9-bb6d-5f1d5451d007 admin admin] PATH_INFO: `/v3/projects/5ea3fcfe484c40eba7e39b2fc9deac2c/users/7fd3c94c5a304ab0a78334d2a46c5f49/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:36.504538 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b61d0b-69e2-41c9-bb6d-5f1d5451d007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea3fcfe484c40eba7e39b2fc9deac2c, user_id=7fd3c94c5a304ab0a78334d2a46c5f49, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:36.508991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b61d0b-69e2-41c9-bb6d-5f1d5451d007 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:36.529303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1441/2905] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:39:36 2022] PUT /identity/v3/projects/5ea3fcfe484c40eba7e39b2fc9deac2c/users/7fd3c94c5a304ab0a78334d2a46c5f49/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:39:36.532865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44509309-fb58-440f-a7dd-2116a1f6384d admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:36.546763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44509309-fb58-440f-a7dd-2116a1f6384d admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:36.650423 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02ca40-aaa5-4a06-ba3a-6483265e7cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:36.650964 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02ca40-aaa5-4a06-ba3a-6483265e7cfe admin admin] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:36.651245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02ca40-aaa5-4a06-ba3a-6483265e7cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:36.651521 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02ca40-aaa5-4a06-ba3a-6483265e7cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:36.652172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02ca40-aaa5-4a06-ba3a-6483265e7cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:36.666552 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02ca40-aaa5-4a06-ba3a-6483265e7cfe admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:36.681875 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1465/2906] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 1 00:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 00:39:36.683193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b61d0b-69e2-41c9-bb6d-5f1d5451d007 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:36.694252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b61d0b-69e2-41c9-bb6d-5f1d5451d007 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:36.699638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac3b95-bc78-4459-90d0-5794de1091e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:36.700647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac3b95-bc78-4459-90d0-5794de1091e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:36.700991 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac3b95-bc78-4459-90d0-5794de1091e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:36.701328 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac3b95-bc78-4459-90d0-5794de1091e2 admin admin] PATH_INFO: `/v3/projects/5ea3fcfe484c40eba7e39b2fc9deac2c/users/7fd3c94c5a304ab0a78334d2a46c5f49/roles/1d817eb4d5c2420ea85c0c6ca25df22a` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:36.703376 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac3b95-bc78-4459-90d0-5794de1091e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea3fcfe484c40eba7e39b2fc9deac2c, user_id=7fd3c94c5a304ab0a78334d2a46c5f49, role_id=1d817eb4d5c2420ea85c0c6ca25df22a)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:36.706187 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac3b95-bc78-4459-90d0-5794de1091e2 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:36.721260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1442/2907] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 1 00:39:36 2022] PUT /identity/v3/projects/5ea3fcfe484c40eba7e39b2fc9deac2c/users/7fd3c94c5a304ab0a78334d2a46c5f49/roles/1d817eb4d5c2420ea85c0c6ca25df22a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:39:41.941631 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd7af3-bc17-4988-9e5f-d480d8ec8fb5 None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:41.942988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd7af3-bc17-4988-9e5f-d480d8ec8fb5 None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:41.942988 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd7af3-bc17-4988-9e5f-d480d8ec8fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:41.978036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-7ffd7af3-bc17-4988-9e5f-d480d8ec8fb5 None None] MFA Rules not processed for user `7fd3c94c5a304ab0a78334d2a46c5f49`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:41.980042 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7ffd7af3-bc17-4988-9e5f-d480d8ec8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:42.058084 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1466/2908] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:39:42.074496 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac3b95-bc78-4459-90d0-5794de1091e2 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:42.077015 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-0bac3b95-bc78-4459-90d0-5794de1091e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:42.225201 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac3b95-bc78-4459-90d0-5794de1091e2 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:42.230794 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186e661-c4cf-45cf-9e9f-f20e2d6482bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:42.231398 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186e661-c4cf-45cf-9e9f-f20e2d6482bc service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:42.231673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186e661-c4cf-45cf-9e9f-f20e2d6482bc service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:42.231943 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186e661-c4cf-45cf-9e9f-f20e2d6482bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:42.242174 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:42.291649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186e661-c4cf-45cf-9e9f-f20e2d6482bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:42.293313 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186e661-c4cf-45cf-9e9f-f20e2d6482bc service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:42.302666 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1443/2909] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:39:44.418923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd7af3-bc17-4988-9e5f-d480d8ec8fb5 None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:44.420801 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-7ffd7af3-bc17-4988-9e5f-d480d8ec8fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:44.463912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:44.489770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd7af3-bc17-4988-9e5f-d480d8ec8fb5 None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:44.493364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c880a-9b96-4c3f-8c19-b24551cc76d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:44.493963 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c880a-9b96-4c3f-8c19-b24551cc76d1 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:44.494233 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c880a-9b96-4c3f-8c19-b24551cc76d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:44.494496 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c880a-9b96-4c3f-8c19-b24551cc76d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:44.496208 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f21c880a-9b96-4c3f-8c19-b24551cc76d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:44.509740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c880a-9b96-4c3f-8c19-b24551cc76d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:44.511972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c880a-9b96-4c3f-8c19-b24551cc76d1 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:44.525449 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1467/2910] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:39:46.902993 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186e661-c4cf-45cf-9e9f-f20e2d6482bc service nova] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:46.913021 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186e661-c4cf-45cf-9e9f-f20e2d6482bc service nova] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:46.917055 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061939d2-ced3-4354-af64-d4848091251a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:46.917758 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061939d2-ced3-4354-af64-d4848091251a service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:46.917930 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061939d2-ced3-4354-af64-d4848091251a service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:46.918202 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061939d2-ced3-4354-af64-d4848091251a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:46.919976 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-061939d2-ced3-4354-af64-d4848091251a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:46.936952 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061939d2-ced3-4354-af64-d4848091251a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:46.938452 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061939d2-ced3-4354-af64-d4848091251a service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:46.952700 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1444/2911] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:39:54.519711 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=11 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:56.180826 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c880a-9b96-4c3f-8c19-b24551cc76d1 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:39:56.189706 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c880a-9b96-4c3f-8c19-b24551cc76d1 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:39:56.193340 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323d97e-c411-46f2-96b9-742dbf85015c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:39:56.193928 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323d97e-c411-46f2-96b9-742dbf85015c service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:56.194202 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323d97e-c411-46f2-96b9-742dbf85015c service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:56.194468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323d97e-c411-46f2-96b9-742dbf85015c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:56.197071 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f323d97e-c411-46f2-96b9-742dbf85015c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:56.260121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323d97e-c411-46f2-96b9-742dbf85015c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:39:56.261652 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323d97e-c411-46f2-96b9-742dbf85015c service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:39:56.271161 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1468/2912] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:39:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:39:56.948177 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=28 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:39:58.650390 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76cc67-082e-404f-9dc1-ed20fb7f3475 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:39:58.651037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76cc67-082e-404f-9dc1-ed20fb7f3475 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:39:58.651037 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76cc67-082e-404f-9dc1-ed20fb7f3475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:39:58.671899 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-aa76cc67-082e-404f-9dc1-ed20fb7f3475 None None] MFA Rules not processed for user `048aba7c5b364b07abf7f6b8977d167a`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:39:58.673167 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-aa76cc67-082e-404f-9dc1-ed20fb7f3475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:39:58.680920 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1445/2913] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:39:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:40:06.266865 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=30 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:40:08.664741 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:40:16.185121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323d97e-c411-46f2-96b9-742dbf85015c service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:40:16.193943 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323d97e-c411-46f2-96b9-742dbf85015c service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:40:16.197912 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d709c-a4bc-4f9f-a377-c246364c6eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:40:16.198507 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d709c-a4bc-4f9f-a377-c246364c6eda service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:40:16.198783 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d709c-a4bc-4f9f-a377-c246364c6eda service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:40:16.199055 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d709c-a4bc-4f9f-a377-c246364c6eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:40:16.207787 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d709c-a4bc-4f9f-a377-c246364c6eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:40:16.209239 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d709c-a4bc-4f9f-a377-c246364c6eda service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:40:16.218577 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1469/2914] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:40:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:40:26.214589 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=6 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:41:03.189945 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76cc67-082e-404f-9dc1-ed20fb7f3475 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:41:03.200123 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76cc67-082e-404f-9dc1-ed20fb7f3475 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:41:03.203946 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8b18a-b4f7-4e23-a8a2-449d8a537cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:41:03.204576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8b18a-b4f7-4e23-a8a2-449d8a537cfe service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:03.204855 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8b18a-b4f7-4e23-a8a2-449d8a537cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:03.205125 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8b18a-b4f7-4e23-a8a2-449d8a537cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:03.206822 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-67d8b18a-b4f7-4e23-a8a2-449d8a537cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:03.216889 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8b18a-b4f7-4e23-a8a2-449d8a537cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:41:03.218408 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8b18a-b4f7-4e23-a8a2-449d8a537cfe service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:41:03.226673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1446/2915] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:41:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:41:12.816995 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d709c-a4bc-4f9f-a377-c246364c6eda service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:41:12.818452 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e79d709c-a4bc-4f9f-a377-c246364c6eda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:12.846136 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d709c-a4bc-4f9f-a377-c246364c6eda service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:41:12.856626 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6367562-23cb-40fc-b764-a368658d1267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:41:12.857676 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6367562-23cb-40fc-b764-a368658d1267 service nova] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:12.858173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6367562-23cb-40fc-b764-a368658d1267 service nova] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:12.858709 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6367562-23cb-40fc-b764-a368658d1267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:12.861776 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-f6367562-23cb-40fc-b764-a368658d1267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:12.885358 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6367562-23cb-40fc-b764-a368658d1267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:41:12.887394 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6367562-23cb-40fc-b764-a368658d1267 service nova] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:41:12.908726 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1470/2916] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:41:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:41:13.222751 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=6 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:41:13.897043 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8b18a-b4f7-4e23-a8a2-449d8a537cfe service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:41:13.911942 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8b18a-b4f7-4e23-a8a2-449d8a537cfe service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:41:13.917845 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb861b-32b3-4c7a-b460-aa8ac2fb281c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:41:13.918749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb861b-32b3-4c7a-b460-aa8ac2fb281c service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:13.919088 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb861b-32b3-4c7a-b460-aa8ac2fb281c service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:13.919417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb861b-32b3-4c7a-b460-aa8ac2fb281c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:13.921435 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-c9bb861b-32b3-4c7a-b460-aa8ac2fb281c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:13.935288 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb861b-32b3-4c7a-b460-aa8ac2fb281c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:41:13.937193 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb861b-32b3-4c7a-b460-aa8ac2fb281c service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:41:13.949178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1447/2917] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:41:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:41:20.185916 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb8c4-da06-4b9a-9af1-d0cae4f25d5c None None] REQUEST_METHOD: `POST` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:20.186364 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb8c4-da06-4b9a-9af1-d0cae4f25d5c None None] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:20.186461 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb8c4-da06-4b9a-9af1-d0cae4f25d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:20.235491 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.auth.core [None req-803bb8c4-da06-4b9a-9af1-d0cae4f25d5c None None] MFA Rules not processed for user `6eb1b515c4a74ba5a9da1c94ec914a75`. Rule list: `[]` (Enabled: `True`). {{(pid=93105) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:41:20.236728 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-803bb8c4-da06-4b9a-9af1-d0cae4f25d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:20.300608 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1471/2918] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 1 00:41:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:41:20.341023 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb861b-32b3-4c7a-b460-aa8ac2fb281c service nova] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:41:20.417252 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb861b-32b3-4c7a-b460-aa8ac2fb281c service nova] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:41:20.422215 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a68a3-e163-4556-a0a7-dd9c1277d736 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6c2a533e849cab240257cc1450c9f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:41:20.422815 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a68a3-e163-4556-a0a7-dd9c1277d736 service swift] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:20.423093 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a68a3-e163-4556-a0a7-dd9c1277d736 service swift] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:20.423361 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a68a3-e163-4556-a0a7-dd9c1277d736 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:20.432224 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a68a3-e163-4556-a0a7-dd9c1277d736 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:41:20.434416 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a68a3-e163-4556-a0a7-dd9c1277d736 service swift] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:41:20.442298 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1448/2919] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 1 00:41:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 00:41:23.981959 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bb8c4-da06-4b9a-9af1-d0cae4f25d5c None None] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:41:23.983393 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-803bb8c4-da06-4b9a-9af1-d0cae4f25d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:24.083661 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bb8c4-da06-4b9a-9af1-d0cae4f25d5c None None] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:41:24.089290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9984ac-6c26-44d1-b4c1-10483125ac28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:41:24.090121 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9984ac-6c26-44d1-b4c1-10483125ac28 service placement] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:24.090252 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9984ac-6c26-44d1-b4c1-10483125ac28 service placement] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:24.090476 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9984ac-6c26-44d1-b4c1-10483125ac28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:24.092343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-2b9984ac-6c26-44d1-b4c1-10483125ac28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:24.104660 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9984ac-6c26-44d1-b4c1-10483125ac28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:41:24.106279 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9984ac-6c26-44d1-b4c1-10483125ac28 service placement] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:41:24.116553 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1472/2920] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:41:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:41:26.960671 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a68a3-e163-4556-a0a7-dd9c1277d736 service swift] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:41:26.962906 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-5a4a68a3-e163-4556-a0a7-dd9c1277d736 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:27.036745 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a68a3-e163-4556-a0a7-dd9c1277d736 service swift] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:41:27.042372 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:41:27.043126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:27.043633 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:27.044203 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] PATH_INFO: `/v3/users/d7396c3776b744328babae60d6cfb6fd` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:27.045518 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7396c3776b744328babae60d6cfb6fd)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:41:27.047550 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:41:27.104768 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7396c3776b744328babae60d6cfb6fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.105481 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7396c3776b744328babae60d6cfb6fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.122662 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7396c3776b744328babae60d6cfb6fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.127099 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:41:27.130576 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7396c3776b744328babae60d6cfb6fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.138302 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7396c3776b744328babae60d6cfb6fd'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.144098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1449/2921] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:41:26 2022] DELETE /identity/v3/users/d7396c3776b744328babae60d6cfb6fd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:41:27.324213 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9984ac-6c26-44d1-b4c1-10483125ac28 service placement] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:41:27.335290 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9984ac-6c26-44d1-b4c1-10483125ac28 service placement] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:41:27.381740 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=100 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:41:27.394043 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:41:27.394693 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:27.394971 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:27.395243 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] PATH_INFO: `/v3/projects/a22a4b4aab87497eb1f8b1bce8cd17d5` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:27.396343 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22a4b4aab87497eb1f8b1bce8cd17d5)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:41:27.398293 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:41:27.443284 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22a4b4aab87497eb1f8b1bce8cd17d5', 'request_id': 'req-e115a1cb-17e6-48b4-9778-d76286c13ef8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.448830 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22a4b4aab87497eb1f8b1bce8cd17d5', 'request_id': 'req-e115a1cb-17e6-48b4-9778-d76286c13ef8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.458761 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22a4b4aab87497eb1f8b1bce8cd17d5', 'request_id': 'req-e115a1cb-17e6-48b4-9778-d76286c13ef8'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.473786 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] The token cache is being invalidate because project a22a4b4aab87497eb1f8b1bce8cd17d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:41:27.474074 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.474701 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.476223 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1473/2922] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:41:27 2022] DELETE /identity/v3/projects/a22a4b4aab87497eb1f8b1bce8cd17d5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:41:27.481085 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:41:27.482592 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:27.495790 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228c76a-97f1-46ea-8fef-499d19d70dd1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:41:27.552724 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:41:27.553350 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:27.553628 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:27.553896 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] PATH_INFO: `/v3/users/c5a0070ed2ef453dae0e93b8ddaf0870` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:27.555012 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a0070ed2ef453dae0e93b8ddaf0870)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:41:27.556997 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:41:27.604239 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a0070ed2ef453dae0e93b8ddaf0870'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.604905 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a0070ed2ef453dae0e93b8ddaf0870'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.616789 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a0070ed2ef453dae0e93b8ddaf0870'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.623638 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a0070ed2ef453dae0e93b8ddaf0870'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.630859 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-ac20b907-5dc7-4215-8fac-ab956e7cc23b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a0070ed2ef453dae0e93b8ddaf0870'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.636839 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1450/2923] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:41:27 2022] DELETE /identity/v3/users/c5a0070ed2ef453dae0e93b8ddaf0870 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:41:27.805896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:41:27.807490 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:41:27.820675 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115a1cb-17e6-48b4-9778-d76286c13ef8 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:41:27.877754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:41:27.878376 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:41:27.878650 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:41:27.878918 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] PATH_INFO: `/v3/projects/af42a4ea101646048c6f453a555b5093` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:41:27.880009 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af42a4ea101646048c6f453a555b5093)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:41:27.881938 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:41:27.928359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af42a4ea101646048c6f453a555b5093', 'request_id': 'req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.934063 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af42a4ea101646048c6f453a555b5093', 'request_id': 'req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.940086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af42a4ea101646048c6f453a555b5093', 'request_id': 'req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.956166 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] The token cache is being invalidate because project af42a4ea101646048c6f453a555b5093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93105) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:41:27.956463 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.957137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:41:27.958932 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1474/2924] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:41:27 2022] DELETE /identity/v3/projects/af42a4ea101646048c6f453a555b5093 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:41:37.633265 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=57,keystone:INSERT=2,keystone:DELETE=11 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:41:37.951309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:42:15.306504 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ada7-772f-488e-b992-dfb4c5cf7f3e None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:42:15.307000 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ada7-772f-488e-b992-dfb4c5cf7f3e None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:42:15.307651 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0ada7-772f-488e-b992-dfb4c5cf7f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:42:15.327696 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-42e0ada7-772f-488e-b992-dfb4c5cf7f3e None None] MFA Rules not processed for user `048aba7c5b364b07abf7f6b8977d167a`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:42:15.328916 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-42e0ada7-772f-488e-b992-dfb4c5cf7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:42:15.382869 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1451/2925] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:42:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:42:16.133734 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:42:16.135254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:42:16.207285 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71afb5e-59a8-4db1-8dfb-d978ddfb910e admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:42:16.212254 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b0824-6227-403d-b34e-b48e68723f6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:42:16.212893 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b0824-6227-403d-b34e-b48e68723f6f service placement] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:42:16.213173 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b0824-6227-403d-b34e-b48e68723f6f service placement] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:42:16.213443 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b0824-6227-403d-b34e-b48e68723f6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:42:16.215160 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-d55b0824-6227-403d-b34e-b48e68723f6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:42:16.226903 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b0824-6227-403d-b34e-b48e68723f6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:42:16.228482 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b0824-6227-403d-b34e-b48e68723f6f service placement] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:42:16.238930 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1475/2926] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:42:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:42:25.376069 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=25 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:42:26.234172 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=32 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:42:29.870008 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0ada7-772f-488e-b992-dfb4c5cf7f3e None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:42:29.871464 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-42e0ada7-772f-488e-b992-dfb4c5cf7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:42:29.947098 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0ada7-772f-488e-b992-dfb4c5cf7f3e None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:42:29.952541 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2f848-c63a-457a-8c83-4cc394eceb6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:42:29.953151 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2f848-c63a-457a-8c83-4cc394eceb6f service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:42:29.953423 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2f848-c63a-457a-8c83-4cc394eceb6f service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:42:29.953692 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2f848-c63a-457a-8c83-4cc394eceb6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:42:29.955417 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e7b2f848-c63a-457a-8c83-4cc394eceb6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:42:29.967474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2f848-c63a-457a-8c83-4cc394eceb6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:42:29.969291 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2f848-c63a-457a-8c83-4cc394eceb6f service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:42:29.980122 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1452/2927] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:42:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:42:39.975882 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=34 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:43:10.515137 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b0824-6227-403d-b34e-b48e68723f6f service placement] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:43:10.525678 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b0824-6227-403d-b34e-b48e68723f6f service placement] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:43:10.530910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:43:10.531495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:43:10.531770 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:43:10.532066 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:43:10.533785 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:43:10.545468 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:43:10.547032 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:43:10.556921 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1476/2928] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:43:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:43:20.552596 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=6 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:44:08.252183 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dec11-944c-44b9-a713-8266fe8afc94 None None] REQUEST_METHOD: `POST` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:44:08.252749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dec11-944c-44b9-a713-8266fe8afc94 None None] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:44:08.252749 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dec11-944c-44b9-a713-8266fe8afc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:44:08.272511 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.auth.core [None req-e31dec11-944c-44b9-a713-8266fe8afc94 None None] MFA Rules not processed for user `048aba7c5b364b07abf7f6b8977d167a`. Rule list: `[]` (Enabled: `True`). {{(pid=93106) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 00:44:08.273708 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-e31dec11-944c-44b9-a713-8266fe8afc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:44:08.280381 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1453/2929] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 1 00:44:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 00:44:08.603124 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:44:08.604583 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:44:08.660539 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a2581-de57-4f75-bbd3-44b523b011b2 service placement] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:44:08.664038 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:44:08.664670 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:44:08.664949 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:44:08.665219 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:44:08.666896 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:44:08.677300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:44:08.678384 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:44:08.686889 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1477/2930] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:44:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 00:44:18.267048 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=3 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:44:18.682869 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=26 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:46:04.756101 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dec11-944c-44b9-a713-8266fe8afc94 None None] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:04.766310 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dec11-944c-44b9-a713-8266fe8afc94 None None] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:04.769852 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f19597-fa0e-450d-8c3e-d02731b824f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:04.770449 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f19597-fa0e-450d-8c3e-d02731b824f2 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:04.770726 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f19597-fa0e-450d-8c3e-d02731b824f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:04.770994 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f19597-fa0e-450d-8c3e-d02731b824f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:04.772744 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-36f19597-fa0e-450d-8c3e-d02731b824f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:04.784100 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f19597-fa0e-450d-8c3e-d02731b824f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:04.785565 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f19597-fa0e-450d-8c3e-d02731b824f2 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:04.795039 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1454/2931] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:46:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:46:05.984300 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:05.985947 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:05.999315 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8becc0c-75fa-441e-ad73-63b42b352d38 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:06.003361 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae2167-7c74-4d0a-a0cd-0e0deab60437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:06.003967 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae2167-7c74-4d0a-a0cd-0e0deab60437 service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:06.004245 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae2167-7c74-4d0a-a0cd-0e0deab60437 service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:06.004613 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae2167-7c74-4d0a-a0cd-0e0deab60437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:06.006309 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a4ae2167-7c74-4d0a-a0cd-0e0deab60437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:06.018493 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae2167-7c74-4d0a-a0cd-0e0deab60437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:06.020036 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae2167-7c74-4d0a-a0cd-0e0deab60437 service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:06.029680 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1478/2932] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:46:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:46:06.117126 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f19597-fa0e-450d-8c3e-d02731b824f2 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:06.126625 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f19597-fa0e-450d-8c3e-d02731b824f2 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:06.130275 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334c824-8b53-47de-97c6-8d2eb4ba1095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:06.130878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334c824-8b53-47de-97c6-8d2eb4ba1095 service neutron] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:06.131154 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334c824-8b53-47de-97c6-8d2eb4ba1095 service neutron] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:06.131425 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334c824-8b53-47de-97c6-8d2eb4ba1095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:06.133171 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-3334c824-8b53-47de-97c6-8d2eb4ba1095 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:06.147260 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334c824-8b53-47de-97c6-8d2eb4ba1095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:06.148649 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334c824-8b53-47de-97c6-8d2eb4ba1095 service neutron] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:06.160156 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1455/2933] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:46:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:46:07.371197 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae2167-7c74-4d0a-a0cd-0e0deab60437 service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:07.380767 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae2167-7c74-4d0a-a0cd-0e0deab60437 service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:07.384419 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9569e-006f-4905-a4e8-983c1583489c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:07.385077 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9569e-006f-4905-a4e8-983c1583489c service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:07.385357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9569e-006f-4905-a4e8-983c1583489c service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:07.385627 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9569e-006f-4905-a4e8-983c1583489c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:07.387389 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-87a9569e-006f-4905-a4e8-983c1583489c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:07.401923 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9569e-006f-4905-a4e8-983c1583489c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:07.403359 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9569e-006f-4905-a4e8-983c1583489c service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:07.415352 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1479/2934] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:46:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:46:15.565503 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334c824-8b53-47de-97c6-8d2eb4ba1095 service neutron] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:15.579844 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334c824-8b53-47de-97c6-8d2eb4ba1095 service neutron] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:15.586458 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c323b6-b82c-41e7-a57b-326d859cef11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb204040ab2f4168b35c3e78bacb849f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:15.587578 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c323b6-b82c-41e7-a57b-326d859cef11 service nova] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:15.588065 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c323b6-b82c-41e7-a57b-326d859cef11 service nova] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:15.588405 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c323b6-b82c-41e7-a57b-326d859cef11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:15.590363 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-07c323b6-b82c-41e7-a57b-326d859cef11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:15.603824 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c323b6-b82c-41e7-a57b-326d859cef11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:15.605572 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c323b6-b82c-41e7-a57b-326d859cef11 service nova] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:15.620238 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1456/2935] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:46:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:46:17.410177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=12 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:46:18.396610 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9569e-006f-4905-a4e8-983c1583489c service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:18.406177 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9569e-006f-4905-a4e8-983c1583489c service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:18.409763 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdf97e9e0340441c87316ce03d46cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:18.410357 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:18.410924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:18.410924 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:18.412702 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:18.476858 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:18.478288 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:18.488410 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1480/2936] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:46:24.599853 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c323b6-b82c-41e7-a57b-326d859cef11 service nova] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:24.611606 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c323b6-b82c-41e7-a57b-326d859cef11 service nova] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:24.616561 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aae6d-2f6b-422a-843f-d6e482f70b4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:24.617535 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aae6d-2f6b-422a-843f-d6e482f70b4a service placement] REQUEST_METHOD: `GET` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:24.617818 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aae6d-2f6b-422a-843f-d6e482f70b4a service placement] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:24.618084 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aae6d-2f6b-422a-843f-d6e482f70b4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:24.619778 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-ff3aae6d-2f6b-422a-843f-d6e482f70b4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:24.636957 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aae6d-2f6b-422a-843f-d6e482f70b4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:24.639721 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aae6d-2f6b-422a-843f-d6e482f70b4a service placement] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:24.656111 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1457/2937] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:46:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:46:24.662727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:24.662727 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:24.674277 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e844975-6442-4a5a-82df-68c93c18b88b service neutron] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:24.681356 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:24.682580 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:24.683119 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:24.683591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] PATH_INFO: `/v3/users/048aba7c5b364b07abf7f6b8977d167a` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:24.685227 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=048aba7c5b364b07abf7f6b8977d167a)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:24.687280 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:24.736879 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '048aba7c5b364b07abf7f6b8977d167a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:24.737617 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '048aba7c5b364b07abf7f6b8977d167a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:24.745502 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '048aba7c5b364b07abf7f6b8977d167a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:24.753956 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '048aba7c5b364b07abf7f6b8977d167a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:24.754649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '048aba7c5b364b07abf7f6b8977d167a'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:24.774317 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1481/2938] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:46:24 2022] DELETE /identity/v3/users/048aba7c5b364b07abf7f6b8977d167a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:46:24.958304 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aae6d-2f6b-422a-843f-d6e482f70b4a service placement] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:24.969004 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aae6d-2f6b-422a-843f-d6e482f70b4a service placement] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:25.028996 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:25.029632 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:25.029914 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:25.030191 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] PATH_INFO: `/v3/projects/31e9903e4f16471d8057ee7364cf123b` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:25.031314 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e9903e4f16471d8057ee7364cf123b)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:25.033278 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:25.093773 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e9903e4f16471d8057ee7364cf123b', 'request_id': 'req-26eeba9f-050c-4f7b-bbd8-9880140c99f1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.099420 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e9903e4f16471d8057ee7364cf123b', 'request_id': 'req-26eeba9f-050c-4f7b-bbd8-9880140c99f1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.106261 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e9903e4f16471d8057ee7364cf123b', 'request_id': 'req-26eeba9f-050c-4f7b-bbd8-9880140c99f1'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.120424 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] The token cache is being invalidate because project 31e9903e4f16471d8057ee7364cf123b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:46:25.120819 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.121509 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.123710 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1458/2939] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:46:24 2022] DELETE /identity/v3/projects/31e9903e4f16471d8057ee7364cf123b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:46:25.127581 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:25.129039 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:25.142900 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093e963-6d31-4e85-8ba2-59cad84a2034 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:25.202495 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:25.203051 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:25.203326 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:25.203591 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] PATH_INFO: `/v3/users/7fd3c94c5a304ab0a78334d2a46c5f49` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:25.204739 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd3c94c5a304ab0a78334d2a46c5f49)` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:25.206649 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:25.212791 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:46:25.255630 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd3c94c5a304ab0a78334d2a46c5f49'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.256276 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd3c94c5a304ab0a78334d2a46c5f49'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.263972 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd3c94c5a304ab0a78334d2a46c5f49'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.271128 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd3c94c5a304ab0a78334d2a46c5f49'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.271754 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.notifications [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd3c94c5a304ab0a78334d2a46c5f49'} {{(pid=93105) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.289164 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1482/2940] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 1 00:46:25 2022] DELETE /identity/v3/users/7fd3c94c5a304ab0a78334d2a46c5f49 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 00:46:25.468388 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] Authenticating user token {{(pid=93106) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:46:25.469857 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.fernet_utils [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93106) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:46:25.485982 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eeba9f-050c-4f7b-bbd8-9880140c99f1 admin admin] Validating token access rules against request {{(pid=93106) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:46:25.524677 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:46:25.550647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6789433c4ccf40c89d963e435a40ba65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f6a8f68ec23146108279fe93631f855a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93106) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:46:25.551303 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:46:25.551579 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:46:25.551851 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] PATH_INFO: `/v3/projects/5ea3fcfe484c40eba7e39b2fc9deac2c` {{(pid=93106) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:46:25.552981 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ea3fcfe484c40eba7e39b2fc9deac2c)` {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:46:25.554878 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] RBAC: Authorization granted {{(pid=93106) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:46:25.601255 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ea3fcfe484c40eba7e39b2fc9deac2c', 'request_id': 'req-10ca0239-259b-44f9-bde5-dd584dbfec5d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.606685 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ea3fcfe484c40eba7e39b2fc9deac2c', 'request_id': 'req-10ca0239-259b-44f9-bde5-dd584dbfec5d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.613358 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ea3fcfe484c40eba7e39b2fc9deac2c', 'request_id': 'req-10ca0239-259b-44f9-bde5-dd584dbfec5d'} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.626474 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] The token cache is being invalidate because project 5ea3fcfe484c40eba7e39b2fc9deac2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93106) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 00:46:25.627178 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.627673 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG keystone.notifications [None req-10ca0239-259b-44f9-bde5-dd584dbfec5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93106) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 00:46:25.629343 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: [pid: 93106|app: 0|req: 1459/2941] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 1 00:46:25 2022] DELETE /identity/v3/projects/5ea3fcfe484c40eba7e39b2fc9deac2c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 00:46:35.285582 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:46:35.622647 dv-cinder-820027-21-iscsi devstack@keystone.service[93106]: DEBUG dbcounter [-] [93106] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93106) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 00:48:16.544671 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] Authenticating user token {{(pid=93105) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 00:48:16.546086 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:48:16.683618 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe949a4-f1b1-4c56-82d5-cd1d5aba7fe6 admin admin] Validating token access rules against request {{(pid=93105) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 00:48:16.741910 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f4379-123f-4847-944c-a0b718270e05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e30caaea6d104a53a8654442c2383b7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed93469862874c75951d95e28929c904', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93105) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 00:48:16.742513 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f4379-123f-4847-944c-a0b718270e05 service placement] REQUEST_METHOD: `GET` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 00:48:16.742796 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f4379-123f-4847-944c-a0b718270e05 service placement] SCRIPT_NAME: `/identity` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 00:48:16.743072 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f4379-123f-4847-944c-a0b718270e05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93105) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 00:48:16.744860 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.fernet_utils [None req-808f4379-123f-4847-944c-a0b718270e05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93105) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 00:48:16.756196 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f4379-123f-4847-944c-a0b718270e05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 00:48:16.757811 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f4379-123f-4847-944c-a0b718270e05 service placement] RBAC: Authorization granted {{(pid=93105) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 00:48:16.767952 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: [pid: 93105|app: 0|req: 1483/2942] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 1 00:48:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 00:48:26.763705 dv-cinder-820027-21-iscsi devstack@keystone.service[93105]: DEBUG dbcounter [-] [93105] Writing DB stats keystone:SELECT=89 {{(pid=93105) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}